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Executive Summary 


The Technology Immersion Pilot (TIP) sets forth a vision for technology immersion in Texas public 
schools. The Texas Education Agency (TEA) originally directed more than $14.5 million in federal 
Title II. Part D monies toward funding a wireless learning environment for high-need middle schools 
through a competitive grant process. A concurrent research project funded by a federal Evaluating 
State Educational Technology Programs grant is evaluating whether student achievement improves 
over time as a result of exposure to technology immersion. The Texas Center for Educational Research 
(TCER) — a non-profit research organization in Austin — is the TEA’s primary partner in this four-year 
endeavor. 

The overarching purpose of the study is to scientifically investigate the effectiveness of technology 
immersion in increasing middle school students’ achievement in core academic subjects as measured 
by the Texas Assessment of Knowledge and Skills (TAKS). Technology immersion encompasses 
multiple components, including a laptop computer for every middle school student and teacher, 
wireless access throughout the campus, online curricular and assessment resources, professional 
development and ongoing pedagogical support for curricular integration of technology resources, and 
technical support to maintain an immersed campus. 

Technology Immersion 

As a way to ensure consistent interpretation of technology immersion and comparability across sites, 
the TEA issued a Request for Qualifications (RFQ) that allowed commercial vendors to apply to 
become providers of technology immersion packages. Successful vendor applicants to the RFQ had to 
include the following six components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus 
to ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as learning tools; 

• Online instructional resources that support the state curriculum in English language arts, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, 
learning, and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through a competitive application and expert-review process, the TEA selected three lead vendors as 
providers of technology immersion packages (Dell Computer Inc., Apple Computer Inc., and Region 1 
Education Service Center [ESC]). Prices for packages varied according to the numbers of students and 
teachers, the type of laptop computer, and the vendor provider. Package costs ranged from about 
$1,100 to $1,600 per student. Of the 22 immersion sites, 6 middle schools selected the Apple package, 
15 selected the Dell package, and 1 school selected the Region 1 ESC package (Dell computer). 




Methodology 


Evaluation Design 

The evaluation employs a quasi-experimental research design, and in the first year, included 22 
experimental and 22 control schools. In the project’s second year, however, the research design was 
modified when two middle schools in one district (one experimental and one control) were lost due to 
damage caused by Hurricane Rita on the Texas Gulf coast. Thus, second-year results (for the 2005-06 
school year) are for the remaining 21 treatment and 21 control schools. A re-analysis of baseline data 
for the new sample revealed that school and student characteristics generally were unchanged and 
differences between comparison groups remained statistically insignificant. 

In the second year, researchers examined the nature of project implementation at the immersion sites. 
Additionally, we gauged the effects of technology immersion on teacher and student mediating 
variables as well as the effects of immersion on students’ reading, mathematics, and writing 
achievement. Research questions arc as follows. 

• How is technology immersion implemented, and what factors arc associated with higher 
implementation levels? 

• What is the effect of technology immersion on teachers and teaching? 

• What is the effect of technology immersion on students and learning? and 

• Does technology immersion affect student achievement? 

The Theoretical Framework for Technology Immersion guides the evaluation. The experimental 
research design allows an estimate of the effects of the intervention, which is the difference between 
the treatment and control groups. The framework postulates a linear sequence of causal relationships. 
First, experimental schools arc to be “immersed” in technology through the introduction of technology 
immersion components. An improved school environment for technology should then lead to teachers 
who have greater technology proficiency, use technology more often for their own professional 
productivity, collaborate more with their peers, have students use technology more in their classrooms, 
and use laptops and digital resources to increase the intellectual challenge of lessons. In turn, these 
improved school and classroom conditions should lead students to greater technology proficiency, 
more opportunities for peer collaboration, greater personal self-direction, more rigorous and authentic 
learning experiences, and stronger engagement in school and learning. Student mediating variables 
presumably contribute to increased academic performance as measured by standardized test scores. In 
the framework, prior student achievement and student, family, and school characteristics exert their 
own influence on learning. 

Participating Sites 

Interested districts and associated middle schools responded to a Request for Application (RFA) 
offered by the TEA in spring 2004 to become technology immersion schools. Applicants had to meet 
eligibility requirements for Title II, Part D funds (i.e., high-need due to children from families with 
incomes below the poverty line, schools identified for improvement, or schools with substantial need 
for technology). Technology immersion schools, selected through the competitive grant process, were 
matched by researchers with control schools on key characteristics, including size, regional location, 
demographics, and student achievement. 

The TIP grants targeted high-need schools, thus nearly 70% of students in the study come from 
economically disadvantaged backgrounds, with many schools in rural or isolated locations. Students 
are ethnically diverse, roughly 58% Hispanic and 7% African American. Middle schools are typically 
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small (402 students, on average), but enrollments vary widely (from 83 to 1,447 students). Although 
schools arc highly concentrated in rural and very small Texas districts, about a third of districts and 
schools are in large cities or suburban locations across the state. 

The second-year study focused on two student cohorts. Cohort 1 included 5,538 seventh graders 
(2,627 immersion, 2,91 1 control) who completed their second project year; Cohort 2 included 5,507 
sixth graders (2,685 immersion, 2,822 control) who finished their first year. Altogether, 1,257 teachers 
participated in the project (604 at immersion and 653 at control campuses). 

Data Collection and Analysis 

Data collection involved a mix of qualitative and quantitative data sources. Researchers conducted site 
visits at each of the middle schools in fall 2004 and spring of 2005 and 2006. For this report, we 
concentrate on site-visit data gathered through observations in a sample of sixth- and seventh-grade 
classrooms (English/language aits, mathematics, social studies, and science). Additional measures 
include annual online teacher surveys and student paper-and-pencil surveys. We also gathered school 
and student demographic, attendance, and achievement data from the Texas Public Education 
Information Management System (PEIMS) and Academic Excellence Indicator System (AEIS), and 
data on student disciplinary actions from schools. 

We used either two- or three-level hierarchical linear models (FILM) to analyze immersion effects on 
teachers’ and students’ perceptions of technology and proficiencies and immersion effects on students’ 
TAKS achievement. Three-level FILM growth modeling estimated the effects of immersion on rates of 
growth for dependent variables across three time points (2004, 2005, and 2006). When only two data 
points were available, we used two-level FILM models to estimate the effects of immersion on 2006 
scores. For two-level FILM models, we calculated effect sizes (ES) in standard deviation units (usually 
Cohen’s d). Effect sizes greater than 0.5 are typically interpreted as large. 0.5 to 0.3 as moderate, 0.3- 
0. 1 as small, and less than 0. 1 as trivial. 

The generalization of findings to a broader population is a study limitation. Compared to Texas 
middle-school students as a whole, students in the sample schools are substantially more Fiispanic and 
less White and African American. Middle schools arc also smaller than the statewide average, and 
schools are located either in small or very small districts or large districts. Additionally, the study 
relies on self-reported data from students and teachers for many outcome variables. Nonetheless, the 
triangulation of evidence from multiple sources (surveys, classroom observations, state demographic 
and test databases, student cohorts) verifies the robustness of findings. 

Major Findings 

Summary of First- and Second-Year Findings 

Our first-year report — Evaluation of the Texas Technology Immersion Pilot: First-Year Results 
(Shapley et al., 2006a) — revealed positive effects of technology immersion on schools, teachers, and 
students. Findings for the second year relative to these same variables are generally consistent with 
first-year results. Steadfast outcomes across two evaluation years and two student cohorts show that 
immersing a middle school in technology produces schools with stronger principal leadership for 
technology, greater teacher collaboration and collective support for technology innovation, and 
stronger parent and community support for technology. Additionally, teachers in immersion schools 
are more technically proficient and use technology more often for their own professional productivity, 
their students use technology more often in core-subject classrooms, and teachers adopt more 
integration-oriented and learner-centered ideologies. Students in immersion schools are more 
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technically proficient, use technology more often for learning, interact more often with their peers in 
small-group activities, and have fewer disciplinary problems than control-group students. 

Also consistent with first-year results, we found no significant effect of technology immersion in the 
second year on student self-directed learning, and we found a significantly negative immersion effect 
on student attendance. Moreover, the availability of technology across two years provided no 
significant increase in the intellectual challenge of immersion teachers’ core-subject lessons. 

First-year findings on academic achievement revealed no statistically significant immersion effects on 
TAKS reading or mathematics scores for Cohort 1, sixth graders. Similarly, second-year results for 
Cohort 1 students (as seventh graders) showed no significant effects of immersion on TAKS reading, 
mathematics, or writing achievement. Likewise, achievement results for Cohort 2 students (sixth 
graders involved in the project for one year) revealed no significant effect of immersion on TAKS 
reading achievement. However, for TAKS mathematics, students in immersion schools who began the 
year with higher math pretest scores had significantly higher mathematics achievement than their 
control-group counterparts. The math achievement gap favoring immersion students over control 
widened as students’ pretest scores increased. Although TAKS score differences between immersion 
and control schools usually did not differ by statistically significant margins, second-year achievement 
Lends, in contrast to first-year results, generally favored technology immersion schools. Additional 
details for second-year outcomes are provided below. 

Major Second-Year Findings 

Effects of Immersion on Teachers and Teaching 

Immersion teachers grew in technology proficiency and in their use of technology for 
professional productivity at significantly faster rates than control teachers. Technology 
immersion accelerated teachers’ growth in meeting the state’s Technology Application Standards. In a 
self-assessment of their technology proficiency across three time points, immersion teachers 
considered themselves to be increasingly more technology literate than control teachers in areas 
involving technology operations and pedagogical skills. Similarly, teachers in immersion schools used 
technology significantly more often for administrative and classroom management puiposes. 

Teachers in immersion schools expressed stronger ideological associations across time with 
technology integration and learner-centered practices. While immersion and control teachers 
initially expressed similar views on instructional practices involving technology, immersion teachers 
changed their instructional beliefs at a significantly more positive rate. Immersion teachers indicated 
that they increasingly employed technology integration actions, such as promoting students’ authentic 
problem solving or critical thinking through technology. Immersion teachers also expressed 
increasingly stronger affiliations with constructivist or learner-centered practices, such as having 
students establish individual learning goals, emphasizing experiential learning, and providing real- 
world experiences. 

Teachers at schools with higher concentrations of student poverty grew in technology 
proficiency and adopted new ideologies at slower rates. Teachers who taught at schools with higher 
student poverty levels grew in technology proficiency and embraced technology integration and 
learner-centered practices at slower rates than their peers in more advantaged schools. Weaker 
supports for implementation at more impoverished immersion schools as well as the characteristics of 
teachers employed in those schools (proportionately more male teachers who were less likely than 
females to embrace innovative methods) may at least partially explain immersion teachers’ progress. 
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Given greater abundance of technology, teachers in immersion schools collaborated more often 
with their peers on technology-related issues than control teachers, and students used technology 
more often in immersion classrooms. Teachers at immersion schools compared to control had a 
significantly steeper growth rate for collaborative interactions with colleagues that supported 
improvements in instructional practices (e.g., developing lesson plans, exchanging information about 
students), as well as for the frequency of their students’ classroom activities involving technology. 
Despite their positive growth trend, statistics indicated that by spring 2006 teachers in immersion 
classrooms had students use various technology resources infrequently (i.e., about once or twice a 
month). While the overall level of classroom technology use was low, practices varied across teachers 
and core-subject areas. 

Availability of technology resources had little, if any, effect on the intellectual challenge of 
immersion teachers’ lessons. Technology immersion’s theorized impact on student achievement 
hinges on technology’s facilitation of more rigorous and authentic learning experiences. Observations 
of core-subject teachers in fall 2004 and spring of 2005 and 2006 revealed no statistically significant 
differences between the intellectual demand of immersion and control teachers’ lessons. Across 
classrooms, lessons generally failed to intellectually challenge students. Observed activities most often 
focused on student acquisition of facts, definitions, and algorithms, and less often centered on writing 
lesson-related communication, constructing knowledge (e.g., synthesizing, explaining), or engaging in 
disciplined inquiry (e.g., investigation, experimental inquiry). 

Effects of Immersion on Students and Learning 

Technology immersion significantly increased students’ technology proficiency and narrowed 
the gap between economically advantaged and disadvantaged students. Immersion students made 
greater progress toward mastery of the Texas Technology Applications standards. Estimated yearly 
growth in proficiency for economically advantaged and disadvantaged immersion students in Cohort 1 
were nearly twice the rates for their control-group counterparts. Consequently, by the end of seventh 
grade, economically disadvantaged students in immersion schools surpassed advantaged control 
students in proficiency. Similarly, for Cohort 2, sixth graders, immersion had a significantly positive 
effect on students’ technology proficiency (ES = 0.30). 

Students in immersion schools used technology significantly more often in core-subject 
classrooms and interacted more frequently with their peers in small groups. Similar to their 
teachers’ reports, Cohort 1 students at immersion schools had a significantly steeper growth trend for 
the frequency of classroom activities with technology than control students. Results for Cohort 2 
students, similarly, revealed significant and practically important differences in classroom activities 
favoring immersion schools (ES = 0.83). Along with greater uses of classroom technology, students in 
immersion schools also had more frequent opportunities to learn with other students in small groups 
and to take a more active learning role. 

Although immersion students used technology more often, classroom observations showed that they 
used technology in rather conventional ways. Observed students most frequently used a word 
processor for writing, learned and practiced skills (typically multi-choice exercises or digitized 
worksheets), created or made presentations (using PowerPoint or Keynote), or conducted Internet 
searches for information on an assigned topic. In general, changes in classroom activities and 
organizational structures in immersion classrooms did not necessarily alter the rigor or relevance of 
students’ experiences with core-subject content. 



Technology immersion had no significant effect on student self-directed learning. We theorized 
that opportunities for independent and self-guided learning afforded through one-to-one technology 
would positively affect students’ personal self-direction. Findings in the second year replicated first- 
year results showing there was no significant immersion effect on self-directed learning. As both 
immersion and control students in Cohort 1 progressed from sixth to seventh grade, their responses to 
statements measuring self-direction revealed a significantly negative growth trend. Results for 
Cohort 2 students, similarly, revealed no significant immersion effect (ES = 0.03). 

Outcomes for student engagement varied. Students in immersion schools had significantly fewer 
disciplinary actions, similar levels of school satisfaction, and significantly lower school 
attendance rates than control-group students. One-to-one computing is often credited with 
increasing student engagement as measured by indicators such as stronger commitment to academic 
work, increased attendance, and reduced discipline problems. Accordingly, interviewed 
administrators, teachers, and students involved in this study have cited greater student interest and 
motivation for school and learning as positive immersion effects. Results for quantitative measures, 
however, were mixed. 

Disciplinary Action Reports for the 2005-06 school year showed that immersion students had 
proportionately fewer behavioral and disciplinary problems than their counterparts in control schools 
(ES = 0.14 and 0.16 for Cohorts 1 and 2, respectively). Conversely, surveys of students’ school 
satisfaction showed no significant differences between immersion and control students’ satisfaction 
with the kinds of work they do in classes or with the relevance of their schoolwork. Unexpectedly, 
technology immersion had a significantly negative effect on school attendance. For Cohort 1 students, 
school attendance rates declined across years, and by the end of seventh grade, the estimated average 
attendance rate for economically advantaged immersion students was 95.9% compared to 96.4% for 
control students (rates were lower for disadvantaged students). Results for Cohort 2 students, 
similarly, showed statistically significant but small differences in attendance rates favoring students in 
control schools (ES = 0.07). 

Effects of Immersion on Academic Achievement 

Technology immersion’s ultimate goal is increasing students’ achievement in core academic subjects 
as measured by state assessments. For analyses reported below, students’ TAKS scale scores were 
standardized and then normalized as T scores with a mean of 50 and a standard deviation of 10. 

Technology immersion had no statistically significant effect on Cohort 1, seventh graders’ 
achievement in reading, mathematics, or writing. For Cohort 1 students, we used three-level FILM 
growth models to estimate mean rates of change in TAKS reading and mathematics scores and a two- 
level FILM model to estimate the effects of immersion on TAKS writing scores. 

• Reading. Controlling for student and school poverty, there was no significant effect of 
immersion on students’ growth rate for TAKS reading. The immersion effect was positive but 
not by a statistically significant margin. Economically disadvantaged students in both 
immersion and control schools grew in reading achievement at a significantly faster rate than 
their more advantaged peers. Combined with the positive immersion result, this yielded a 
positive boost in reading achievement for disadvantaged immersion students. 

• Mathematics. After controls for student and school poverty, there was no significant effect of 
immersion on students’ growth rate for TAKS mathematics. The immersion effect was 
positive but not by a statistically significant margin. In contrast to reading, economically 
disadvantaged students at both immersion and control schools grew in mathematics 
achievement at a significantly slower rate than their more advantaged peers. 
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• Writing. After adjusting for Cohort 1 students’ initial TAKS writing scores (as fourth graders 
in 2003), student demographic characteristics, and school poverty, there was no statistically 
significant difference in the 2006 writing scores for students in immersion and control schools. 
The immersion effect was negative but not by a statistically significant margin. 

Technology immersion had no statistically significant effect on Cohort 2, sixth graders’ reading 
achievement. However, immersion had a significantly positive effect on mathematics scores for 
higher achieving students. We analyzed the effects of immersion on Cohort 2 students’ TAKS 
reading and mathematics scores using two-level HLM models. 

• Reading. Controlling for students’ prior achievement (as fifth graders in 2005), demographic 
characteristics, and school poverty, there was no statistically significant difference in the 2006 
TAKS reading scores for students in immersion and control schools. The immersion effect on 
reading was positive but not by a statistically significant margin. 

• Mathematics. After controls for students’ prior achievement (as fifth graders in 2005), 
demographic characteristics, and school poverty, there was no overall significant difference 
between immersion and control students’ TAKS mathematics scores. The immersion effect 
was positive but not by a statistically significant margin. However, there was a statistically 
significant immersion effect on mathematics achievement that acted through students’ pretest 
scores. Other factors being equal, having higher pretest scores predicted larger gaps in 2006 
math scores favoring immersion students. Thus, immersion had a significantly positive effect 
on mathematics achievement for higher achieving sixth graders. 

Second-year achievement trends generally favored technology immersion schools. Although 
TAKS scores for immersion and control students usually did not differ by statistically significant 
margins in the second year, noteworthy achievement trends emerged. In the first project year, TAKS 
reading and mathematics achievement trends favored control schools. Conversely, in the second year, 
immersion schools had more positive achievement trends than control schools across both Cohorts 1 
and 2 and for both reading and mathematics subject areas. Outcomes for TAKS writing, in contrast, 
favored students in control schools. The analysis of writing achievement, however, differed from other 
subject areas in the wider span of time between the pretest (4th grade) and posttest (7th grade). The 
testing mode for writing could also have affected outcomes. Immersion students who regularly use 
word processors for writing may be at a disadvantage when completing a writing assessment in 
traditional paper-and-pencil format. 

Second-year findings provide formative evaluation outcomes. The evaluation of technology 
immersion is a four-year, longitudinal study, and findings from the second year provide preliminary 
outcomes. In designing the study, we thought that some effects might emerge during early 
implementation, but we also believed that changes in longer term outcomes, such as student 
achievement, might require at least three years to surface (i.e., time for Cohort 1 students to progress 
from sixth to eighth grade). Additionally, outcomes so far have focused mainly on TAKS reading and 
mathematics. In the third year, Cohort 1, eighth graders will complete TAKS social studies and 
science assessments. Thus, outcomes will be available for each of the core-subject areas. 

Moreover, while student achievement results as measured by TAKS scores are extremely important, 
there are other outcomes for immersion students that may contribute to their long-term success. 
Certainly, technology immersion has narrowed the technology equity gap for economically 
disadvantaged students. Many students who previously had no technology in their homes arc 
becoming computer literate through their experiences with laptops. Administrators, teachers, and 
students alike believe that middle school students at immersion schools are better prepared for future 
educational and workforce requirements and for 21st Century expectations, such as communication 
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skills, and information and media literacy. In the sections to follow, we describe how the generally 
low levels of implementation may have contributed to second-year results. 


Nature of Second-Year Implementation 

Most of the middle schools struggled in the second year to implement the prescribed components 
of technology immersion. Full implementation of the immersion model requires support in several 
ways: Leadership, Teacher Support (buy-in), Parent and Community Support, Technical Support, and 
Professional Development. Given adequate supports, teachers are expected to reach high levels of 
Classroom Immersion, and Student Access and Use of technology is expected to be robust. The 
Implementation Index, a composite campus score measuring the strength of immersion components, 
showed that a third of middle schools (6 of 21) attained a stronger presence of components that nearly 
approximated expected standards {substantial immersion ), whereas two-thirds of schools had lower 
implementation levels {minimal to partial immersion). Overall, mean immersion standard scores 
(ranging from 2.48 to 3.06) indicated that supports for immersion generally failed to meet full 
implementation standards (3.50 to 4.00). With mainly low-to-moderate supports, the average levels of 
Classroom Immersion (2.48) and Student Access and Use (2.17) were below expectations. Major 
concerns included students’ inconsistent use of laptops across classrooms and subject areas, uneven 
provision of professional development supporting the design of effective technology-infused lessons, 
and variability in students’ access to laptops during the school day and at home. 

The strength of professional development and other supports were associated with higher levels 
of classroom and student immersion. Variability in the quality of professional development 
provided by schools was a major obstacle to teachers’ growth in creating technology-immersed 
classrooms. While the immersion model requires that a quarter of grant funds be expended for 
professional development, the design rested largely with individual districts and campuses and their 
selected technology vendors (mainly Apple or Dell). Our measure of the strength of the campus 
professional development component was significantly correlated with teachers’ reported levels of 
classroom immersion. Leadership for immersion also emerged as an important factor in advancing 
change. Principals appeared to influence teachers’ attitudes toward technology through their provision 
of supports for changed practice. Similarly, students’ access to and use of technology for learning was 
significantly related to their teachers’ greater involvement in professional development and the 
strength of other school supports for immersion. 

A continuing challenge in the second year was the consistent provision of laptops for students 
both within and outside of school. Student laptop access varied widely both across and within 
schools. The average number of laptop access days reported by students ranged from 42 to 178 days, 
with only a few campuses achieving full access (the targeted 170 to 180 days per student). Student 
laptop access was limited by factors such as disciplinary infractions, technical issues, time for repairs, 
and in a few cases, parent resistance. Additionally, some immersion schools allowed students to have 
unlimited access to laptops outside of the school day, while others restricted students’ out-of-school 
access to a series of days or to laptop check-outs for teacher-assigned schoolwork. Overall, laptops’ 
potential influence on learning varied across students and schools. 

Schools with a greater proportion of economically disadvantaged students had lower 
implementation levels. Schools with larger concentrations of student poverty had significantly lower 
levels of implementation. Accordingly, teachers at these schools grew in proficiency and created 
immersed classrooms at significantly slower rates than teachers in more advantaged schools. Schools 
serving predominantly disadvantaged and often low-performing student populations faced special 
challenges in implementing a project requiring profound school and classroom change. 
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1. Introduction 


The Technology Immersion Pilot (TIP) sets forth a vision for technology immersion in Texas public 
schools. The Texas Education Agency (TEA) originally directed more than $14.5 million in federal 
Title II, Part D monies toward funding a wireless learning environment for high-need middle schools 
through a competitive grant process. A concurrent research project funded by a federal Evaluating 
State Educational Technology Programs grant is evaluating whether student achievement improves 
over time as a result of exposure to technology immersion. The Texas Center for Educational Research 
(TCER) — a non-profit research organization in Austin — is the TEA’s primary partner in this four-year 
endeavor. 

The overarching purpose of the study is to scientifically investigate the effectiveness of technology 
immersion in increasing middle school students’ achievement in core academic subjects as measured 
by the Texas Assessment of Knowledge and Skills (TAKS). Technology immersion encompasses 
multiple components, including a laptop computer for every middle school student and teacher, 
wireless access throughout the campus, online curricular and assessment resources, professional 
development and ongoing pedagogical support for curricular integration of technology resources, and 
technical support to maintain an immersed campus. The evaluation also aims to examine the 
relationships that exist among contextual conditions, technology immersion, intervening factors 
(school, teacher, and student), and student achievement. In the first year, there were 22 experimental 
and 22 control sites in the study. However, in the second year, two middle schools in one school 
district (one experimental and one control) were removed from the study due to the devastating effects 
of Hurricane Rita on the Texas Gulf coast. School buildings were damaged, laptops destroyed, and the 
school year disrupted. Thus, second-year results arc for remaining 21 experimental and 21 control 
schools. 


Theory of Technology Immersion 

In recent years, the vision for educational technology endorsed by many educators, leaders, and 
policymakers has shifted from the use of particular technology software products to technology’s 
incorporation into every aspect of the educational environment. Changing views reflect our growing 
understanding of how students learn and how to create technology -infused environments that enhance 
teaching and learning. Cognitive science and other research reveal that children learn more when they 
are engaged in meaningful, relevant, and intellectually stimulating work (Bransford, Brown, & 
Cocking, 2003; Newmann, Bryk, & Nagoaka, 2001). Many also believe that educational technology 
can help students develop the competencies needed for the 21st century. Children who are growing up 
in the Digital Age must have different competencies, including digital literacy, inventive thinking, and 
effective communication (CEO Forum, 2001; Lempke, Couglin, Thandani, & Martin, 2003). 
Correspondingly, there is a growing concern that U.S. schools are not preparing students to succeed in 
the modern world. Today’s graduates must be critical thinkers, problem solvers, and effective 
communicators who are proficient in core subjects as well as in information and media literacy 
(Partnership for 21 st Century Skills, 2006). 

Similarly, Texas has long recognized that the state’s success is tied to the provision of opportunities 
for the Digital Age. Preparing for the 21st century means that Texas students must learn different ways 
to work with tools, information, and people. The Texas Long-Range Plan for Technology, 2006-2020, 
advances the previous state plan’s approach for the integration of technology within schools across 


1 




four major domains: teaching and learning; educator preparation and development; leadership, 
administration, and instructional support; and infrastructure for technology (TEA, 2006). Texas Senate 
Bill 396, enacted during the 2003 Texas legislative session, further defines this comprehensive 
vision as “technology immersion.” Technology immersion calls for the provision of a wireless mobile 
computing device for each student in a school, the use of technology-based learning resources, training 
teachers to integrate technology into the classroom, and the provision of support for effective 
technology use. Consistent with the overall Texas vision for technology, the long-term aspiration for 
technology immersion is to “prepare each student for success and productivity as a lifelong learner, a 
world-class communicator, a competitive and creative knowledge worker, and an engaged and 
contributing member of an emerging global society” (TEA, 2006, p. viii). 

Technology Immersion Components 

While Texas state statutes provide a general description of technology immersion, TEA staff relied on 
existing research on educational technology as well as practical wisdom gained through numerous 
pilot studies and statewide technology initiatives to specify critical components of immersion. The 
technology immersion model assumes that effective technology use in schools and classrooms requires 
(a) robust technology access, (b) technical and pedagogical support for implementation, 

(c) professional development for educators in using technology effectively, and (d) readily available 
curricular and assessment resources that support the state’s curriculum in the core subjects (English 
language arts, mathematics, science, and social studies). 

Robust Access to Technology 

The targeted level of technology access in Texas is one-to-one access to Internet-connected 
multimedia computers in classrooms. Similar to national trends, the Texas ratio of students to 
instructional computers was 3.5 to 1 in 2006, a slight decrease from the ratio of 3.7 to 1 in 2001. Texas 
schools also have built their infrastructure for technology. Texas had a 3.4 ratio of students per high- 
speed Internet-connected computers in 2006 (Education Week, 2007). Despite school-level 
improvements in technology access across years, a statewide survey conducted in 2002 and baseline 
data collected for this study in 2004 indicate that an average of 2.9 or less classroom computers is 
insufficient to allow every student access (Shapley, Benner, Heikes, & Pieper, 2002; Shapley et al., 
2006b). Correspondingly, when Texas middle-school students in 2005 were asked how they found out 
about websites and new technology and how to use them, only 13% reported that they learned from 
“teachers or classes in school;” students, instead, indicated they learned from “my friends” (32%) or “I 
explore on my own” (34%) (NetDay, 2005). 

Additionally, inequities in technology access continue to pose challenges for economically 
disadvantaged and minority students both nationally and in Texas. While access to computers and the 
Internet in higher and lower poverty schools has narrowed in recent years, the income-related gap 
persists. Nationally, the rate of at-home computer use among children from families earning less than 
$20,000 a year was 37% percent compared to 80% or higher for children from families earning 
$50,000 or more (Trotter, 2007). Thus, low-income students had fewer opportunities than their more 
advantaged peers to develop effective technology skills and to enhance learning at home. Likewise, 
minority and economically disadvantaged students in Texas arc less often exposed to technology 
outside of school (Shapley et al., 2002). 

As a way to counteract prevailing conditions, technology immersion aims for one-to-one student 
technology access. The Texas project is not unique in its quest for one-to-one computing. As computer 
technologies have become more affordable and accessible, large-scale projects have begun to appear 
with each student in a school, grade level, or classroom receiving his or her own computing device 
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(Zucker, 2004; Penuel, 2006). Although the technology immersion pilot is si mi lar to other laptop 
projects in its provision of one-to-one computing, it is unique in its focus on immersing entire schools 
in technology and simultaneously providing implementation supports. 

Technical and Pedagogical Support 

Technology immersion also assumes that increased access to and use of technology in schools requires 
a healthy technical infrastructure and adequate technical and pedagogical support. Schools must have 
electronic networks that arc robust enough to support wireless laptops and digital content. Campus- 
based technical support is also vital, as many studies emphasize the importance of on-site access to 
support personnel who are responsible for assisting teachers in learning to use technology, 
troubleshooting technical problems, and effectively integrating technology into lessons (e.g., CEO 
Forum, 2001; Ringstaff & Kelley, 2002; Shapley et al., 2002). Studies have found a strong relationship 
between the provision of quality technology support and teachers’ technology use and their changes in 
use over time (Ronnkvist, Dexter, & Anderson, 2000; National Center for Education Statistics 
[NCES], 2000). Disparities in access to technology support also emerge. Teachers at low- 
socioeconomic schools and at smaller schools and districts report less technical and instructional 
support (Ronnkvist et al., 2000; Shapley et al., 2002). In addition to technical assistance, ongoing 
professional development and pedagogical support for teachers’ efforts to use technology, as discussed 
below, is crucial. Considering the importance of support for implementation, technology immersion 
requires that each school provide technical assistance and ongoing pedagogical support. 

Professional Development 

Technology immersion assumes that technology’s potential impact on student learning depends on 
teachers’ opportunities for effective professional development. Research shows that effective 
professional development should be of appropriate duration, provide ongoing support, be relevant to 
individual needs, entail active learning, build content knowledge, and contribute to a professional 
culture (e.g., Hawley & Valli, 1999; Newmann & Associates, 1996; Garet, Porter, Desimone, Birman, 
& Yoon, 2001). In particular, research shows that professional development activities of longer 
duration provide richer learning experiences (Garet et al., 2001). For technology, training should be of 
an adequate length to comprehensively investigate the topics and provide time for practice and 
experimentation (American Council on Education, 1999; Lewis, et al., 1999; Smerdon, et al., 2000). 
Evidence shows that when a particular - technology use is mastered by teachers over time or promoted 
through sustained professional development, it is more likely to be incorporated into instruction (Zhao 
& Frank, 2003). 

Professional development also should include follow-up to support teachers as they acquire and 
implement new skills in the instructional setting (Apple Computer, Inc., 1995; Garet et al., 2001). 
While structured professional development provides a start, ongoing, campus-based mentoring and 
coaching is also necessary to help teachers learn try out new technology-based instruction and 
activities in the classroom (Bradburn & Osborne, 2007; Nugent & Fox, 2007; Sulla, 1999). Effective 
professional development should also focus on subject-specific content or specific teaching methods. 
For technology, this means that activities should not just build teachers’ basic technology skills but 
should support their understanding of effective curricular integration methods as well (CEO Forum, 
2000, 2001; Denton, Davis, & Strader, 2001; Ringstaff & Kelly, 2002; Web-Based Education 
Commission, 2000). 

Additionally, technology professional development should not be isolated but should be part of 
broader professional growth initiatives in schools (Fullan & Hargreaves, 1996; Mann, Shakeshaft, 
Becker, & Kottkamp, 1999). Professional development activities that include collective participation 
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(e.g., whole schools or teachers of the same subjects or grades) are more likely to be coherent with 
teachers’ experiences and needs (Garet et al., 2001). Through collective experiences, teachers develop 
shared norms and values that together reinforce new practices (Newmann & Associates, 1996). A 
leadership development component is also vital. Research points consistently to the important role of 
school leaders in successful implementation of technology (Bradburn & Osborne, 2007 ; Johnston & 
Cooley, 2001; Pitler, 2005). 

Teacher involvement in technology-related professional development also has been associated with 
positive outcomes. Teachers who participate in professional development more often use technology 
for instructional puiposes (Becker, 1999; Kanaya, Light, & Culp, 2005; Martin & Shulman, 2006; 
NCES, 2002; Wenglinsky, 1998). Moreover, as training participation increases, teacher reports of 
feeling well prepared to use technology for instruction increase as well (Smerdon et al., 2000). 

Curricular and Assessment Resources 

Technology’s impact on student academic achievement in an immersed school hinges on the 
availability of instructional and learning resources that support the state’s curriculum. Immersion 
resources include productivity, communication, and presentation software that allow students and 
educators to use wireless laptops as a tool for teaching, learning, communication, and productivity. 
Additionally, digital resources (e.g., online, CD-ROMS, stored on local networks) provide a means to 
support more engaged, thoughtful, relevant, and personalized learning activities for students. 
Interactive technologies allow students to build new knowledge by doing, receiving feedback, and 
refining their understanding. Technologies may also help students to acquire more information, 
visualize difficult-to-understand concepts, and advance understanding. Immersion resources, thus, 
provide a means to extend, supplement, or enhance the state’s curriculum. In addition to instructional 
resources, technology immersion provides online formative assessments that allow teachers to 
diagnose students’ strengths and needs and assess progress toward curricular mastery. 

Implementing digital resources aligned with the Texas curriculum is expected to modify existing 
instructional practices. Yet, as others have pointed out, the availability of wireless laptops and digital 
resources may not improve student learning and achievement if teachers fail to use resources or simply 
provide the same kinds of lessons and assignments electronically instead of using new technologies to 
transform students’ learning experiences (Means, Haertel, & Moses, 2003). 

Theoretical Framework for Technology Immersion 

The Theoretical Framework for Technology Immersion guides the evaluation (see Figure 1.1). The 
experimental design, as illustrated in the framework, allows an estimate of the effects of the 
intervention, which is the difference between the experimental and control groups. The framework 
also postulates a linear sequence of causal relationships. Program implementation comes first. 
Experimental schools are to be “immersed” in technology through the introduction of technology 
immersion components. The quality of implementation reflects the robustness of wireless laptop 
access for teachers and students, the adequacy of technical and pedagogical support services to 
maintain an immersed campus, the extent to which professional development supports curricular 
integration of technology, and how well curricular resources and assessments are used. Given quality 
implementation, we expect school-level improvements in measures of classroom technology 
integration, technical support, innovative culture (teacher support or buy-in), and parent and 
community support. Leadership drives progress toward full immersion. 
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Figure 1.1. Theoretical Framework for Technology Immersion 
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An improved school environment for technology should lead to teachers who have greater 
technology proficiency, use technology more often for their own professional productivity, 
collaborate more with their peers, have students use technology more and in new ways in their 
classrooms, and use laptops and digital resources to increase the intellectual challenge of lessons. 
In turn, improved school and classroom conditions should lead students to greater technology 
proficiency and personal self-direction, more frequent classroom technology activities and 
opportunities for peer collaboration, and stronger engagement in school and learning. Student 
mediating valuables presumably contribute to increased academic performance as measured by 
standardized test scores. In the framework, links also are shown between student achievement and 
student, family, and school characteristics, which exert their own influence on learning. 

The study’s theoretical framework has guided the evaluation design as well as the design of data 
collection procedures and measures. The research literature underpinning the framework is 
included in Appendix A. 


Study Questions 

The evaluation of technology immersion employs a quasi-experimental research design with 
middle schools assigned to either treatment or control groups. In the second year, researchers 
answered the following questions: 

• How is technology immersion implemented, and what factors are associated with higher 
implementation levels? 

• What is the effect of technology immersion on teachers and teaching? 

• What is the effect of technology immersion on students and learning? and 

• Does technology immersion affect student achievement? 

This report concentrates on information gathered from the participating middle school campuses 
during the 2005-06 school year. Data collection involved a mix of qualitative and quantitative 
data sources. Researchers conducted site visits in each of the middle schools in fall 2004 and 
again in spring of 2005 and 2006. For this report, we concentrate on data gathered through 
observations in a sample of sixth- and seventh-grade classrooms. Additional measures include 
annual online teacher surveys and student paper-and-pencil surveys. We also gathered school and 
student data from the Texas Public Education Information Management System (PEIMS), the 
Academic Excellence Indicator System (AEIS), and data on student disciplinary actions from 
schools. 


Organization of the Report 

Report sections arc organized around findings relative to the study’s research questions. An 
overview of report chapters is provided below. 

• Chapter 1, Introduction , provides background on the technology immersion project as 
well as the study’s theoretical framework. The section also establishes the puipose for 
the study and the research questions addressed. 

• Chapter 2, Methodology, presents information on the evaluation design, characteristics 
of immersion and control schools, study limitations, study participants, data collection 
methods, and data analysis procedures. 
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• Chapter 3, Technology Immersion — Second-Year Implementation , describes progress 
toward implementation in the second year and factors associated with higher 
implementation. 

• Chapter 4, Effects of Technology Immersion on Teachers and Teaching, presents 
findings on the effects of immersion on teacher variables, including technology 
proficiency, professional productivity, teachers’ ideologies, students’ classroom 
activities with technology, collaboration with peers, and the intellectual challenge of 
lessons. 

• Chapter 5, Effects of Technology Immersion on Students and Learning, offers findings 
on the effects of immersion on mediating variables, including students’ self-perceptions 
of their technology proficiency, self-directed learning, and school satisfaction; students’ 
experiences with technology; and students’ engagement. 

• Chapter 6, Effect of Technology Immersion on Studen t Achievement, presents findings 
on the effects of technology immersion on academic achievement, as measured by 
TAKS reading, mathematics, and writing assessments for Cohorts 1 and 2 students. 

• Chapter 7, Conclusions and Implications. The final section presents the major findings 
from the study and discusses the implications of outcomes. 
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2. Methodology 


Evaluation Design 

The evaluation design is quasi-experimental with a carefully matched comparison group. The design 
aims to approximate a randomly assigned control group by matching immersion schools with non- 
immersion schools possessing similar pre-intervention characteristics. For this study, interested 
districts and associated middle schools responded to a Request for Application (RFA) offered by the 
Texas Education Agency (TEA) to become Technology Immersion Pilot (TIP) schools. Applicants 
had to meet eligibility requirements for Title II, Paid D funds (i.e., high-need due to children from 
families with incomes below the poverty line, schools identified for improvement, or schools with 
substantial need for technology). 

Twenty-two technology immersion schools, selected through the competitive grant process, were 
matched by researchers with 22 control schools on key characteristics, including size, regional 
location, demographics, and student achievement. The TIP grants targeted high-need schools, thus 
nearly 70% of students in the study come from economically disadvantaged backgrounds, with many 
schools in rural or isolated locations. Students are ethnically diverse, roughly 58% Hispanic and 7% 
African American. As noted in the introduction, two middle schools from one district (one treatment 
and one control) were removed from the study in the second year due to the damaging effects of 
Hurricane Rita. Therefore, second-year results are for the remaining 2 1 treatment and 2 1 control 
schools. A re-analysis of baseline data for the new sample revealed that school and student 
characteristics generally were unchanged and differences between groups remained statistically 
insignificant. Thus, the study’s research design appears sound. 

Treatment Sample 

In spring 2004, the TEA released a series of Requests for Applications (RFAs) inviting school districts 
to apply for TIP grants for up to two middle schools. The agency held an external review of proposals, 
with applications scored by five readers and scores rank ordered. Following the external review, 
researchers and agency staff reviewed proposals to ensure that applications met criteria established for 
technology immersion. Final selection of TIP schools involved the consideration of several factors, 
including proposal ratings, size, location, student diversity, and academic achievement. Decisions 
were influenced by the need for geographic distribution and the availability of comparable schools for 
the control group pool. 

Control Sample 

The selection of control campuses first involved the generation of a pool of grades 6-8 middle schools 
eligible to receive federal funds for participation in the study. As a next step, researchers identified 
middle schools that matched treatment campuses as nearly as possible on factors, including (a) district 
and campus size, (b) regional location, (c) the proportion of economically disadvantaged and minority 
students, (d) percentage of students passing all TAKS tests, and (e) the gaps between the percentage of 
White students and African American and Hispanic students passing TAKS (all tests). Selection 
involved the use of SPSS ® statistical software procedures to establish parameters around each variable 
of interest and the creation of a computer-generated list of “best matches” for each treatment school. 
The final selection involved a review of the matched list by a team of six researchers to identify the 
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optimal control school for each treatment school. Additional schools were selected as alternates in the 
case that a selected control site declined the invitation to participate in the study. This selection 
process yielded 22 control group schools including controls for 8 campuses that came from within the 
same districts as the treatment schools and controls for 14 campuses from closely matched single, 
middle school districts. Each control school receives $50,000 annually for study participation, with 
25% of funds earmarked for professional development as required by Title II, Part D guidelines. 


Characteristics of Participating Schools 

The schools participating in the study are compared in Table 2.1. The distribution of middle schools 
across campus and district enrollment categories shows the comparability of treatment and control 
groups. For both groups, middle schools are typically small (enrolling 600 students or less), and they 
are located either in small or very small districts (enrolling 2,999 students or less) or large districts 
(enrolling 10,000 students or more). 


Table 2.1. Campus and District Enrollment by Comparison Group 



Immersion A=21 

Control ;V=2 1 

Number of students 

Number 

Percent 

Number 

Percent 

Campus 

300 or less 

12 

57.1 

12 

57.1 

301-600 

5 

23.8 

4 

19.0 

601 or more 

4 

19.0 

5 

23.8 

District 

999 or less 

8 

38.1 

8 

38.1 

1,000-2,999 

6 

28.6 

5 

23.8 

3,000-9,999 

0 

0.0 

0 

0.0 

10,000 or more 

7 

33.3 

8 

38.1 


Note. Two campuses (one experimental and one control) were excluded from the 
comparison groups in the second year. 


Information in Table 2.2 compares the baseline characteristics of immersion and control schools. 
Results for Mests show that the percentages of economically disadvantaged, minority, English as a 
second language (ESL), and special education students are statistically equivalent across the treatment 
and control schools. Likewise, results for student enrollment, mobility, and TAKS passing rates show 
no significant differences. Consequently, the treatment and control schools are sufficiently well 
matched on key demographic and academic performance measures. Moreover, both treatment and 
control samples include a range of campus and district enrollments and schools from diverse regions 
of the state. In these respects, the sample selection process and matching procedures appear to have 
produced a sample of schools with good internal validity, in that there are no large, statistically 
significant treatment-control differences. Still, the tendency for immersion schools to enroll greater 
proportions of minority and economically disadvantaged students may be important considering 
known links between disadvantaged status and lower achievement. 
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Table 2.2. Comparison of Baseline Characteristics: Technology Immersion (N = 21) and Control 
Schools (N = 21) 

I 95% Confidence Interval for Difference 


Variable 

Condition 

Mean 

SD 

Lower 

Upper 

t (40) 

Enrollment 

Immersion 

374.9 

348.4 

-284.6 

177.5 

-0.47 


Control 

428.5 

391.3 




Economic disadvantage (%) 

Immersion 

70.8 

17.5 

-3.4 

19.4 

1.42 


Control 

62.8 

19.0 




Minority (%) 

Immersion 

68.1 

28.4 

-10.4 

24.7 

0.83 


Control 

60.9 

27.8 




ESL (%) 

Immersion 

13.5 

17.2 

-1.6 

16.0 

1.66 


Control 

6.3 

9.9 




Special education (%) 

Immersion 

14.7 

5.5 

-4.0 

1.8 

-0.76 


Control 

15.8 

3.7 




Student mobility (%) 

Immersion 

15.8 

4.6 

-3.8 

2.8 

-0.30 


Control 

16.3 

5.9 




TAKS 2004, Passing All (%) 

Immersion 

52.4 

15.7 

-9.2 

8.5 

-0.08 


Control 

52.8 

12.5 




TAKS 2003, Passing All (%) 

Immersion 

65.9 

11.4 

-9.1 

5.5 

-0.50 


Control 

67.6 

12.0 





Source: Texas Education Agency AEIS reports 2004 

Note. TAKS = Texas Assessment of Knowledge and Skills. Differences between groups are statistically 
insignificant. Two campuses (one experimental and one control) were excluded from the groups in the second year. 


Table 2.3 provides campus-level data for each of the 42 schools included in the study. Again, data in 
the table show that the treatment and control schools are reasonably well matched on baseline 
characteristics. Middle schools are highly concentrated in rural and very small districts across the 
state. Still, over a third of districts and schools are in large cities or suburban locations in or around 
cities. The sample also includes campus charter schools (one each for the treatment and control group) 
located in a major urban district. 

The primary limitation of the study is external validity — the extent to which the results of an 
experiment can be generalized from the specific sample to the general population. Schools eligible to 
become part of the treatment group were limited to those serving children from families living in 
poverty 1 and middle schools with grades 6 to 8. Only schools that applied for the grant, and submitted 
applications that met a threshold of quality, were eligible for consideration. Due to these restrictions, 
the treatment group is not representative of the average middle school in Texas. 

The majority of students in the sample are economically disadvantaged. The percentage of sample 
students who qualify for federal free or reduced-price lunch exceeds the state average for middle 
schools (67% vs. 51%). The sample also is substantially more Hispanic and less White and African 
American than state middle-school students as a whole. Overall, about 58% of sample students arc 
Hispanic compared to about 37% of Texas middle school students. Conversely, the sample includes 
fewer African American students (7% vs. 14%) and white students (36% vs. 46%) compared to the 
state averages. 


1 Federal definition used: 27% of population or more than 2,500 people living below poverty line. 


11 




Table 2.3. Characteristics of Technology Immersion and Matched Control Schools 
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Table 2.3. Characteristics of Technology Immersion and Matched Control Schools ( Continued ) 
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The sample schools also differ structurally from Texas middle schools as a whole. Middle schools in 
Texas, on average, enroll more students (667 vs. 402 in sample schools) and are concentrated in larger 
districts (11,575 students enrolled, on average, vs. 3,672 in sample schools). Thus, compared to the 
state overall, sample schools and the districts they reside in are smaller and serve more economically 
disadvantaged and Hispanic students. Differences almost certainly reflect funding restrictions (Title II, 
Part D) and the amount of available funds per grant. The maximum grant amount ($750,000) fell well 
short of dollars required to support one-to-one technology in larger middle schools. 

Participants 


Students 

Table 2.4 shows that four groups or cohorts of students will be followed in this study, with Cohort 1 
followed for four years, Cohort 2 for three years, Cohort 3 for two years, and Cohort 4 for one year. 
Data collection activities in 2005-06 centered on Cohorts 1 and 2. 

• Cohort 1 (seventh graders) included a total of 5,538 students, with 2,627 students enrolled at 
treatment campuses and 2,91 1 at control campuses; and 

• Cohort 2 (sixth graders) included a total of 5,507 students, with 2,685 students enrolled at 
treatment campuses and 2,822 at control campuses. 


Table 2.4. Student Cohorts by School Year and Grade 




Middle School 


High School 

Year 

Grade 6 

Grade 7 

Grade 8 

Grade 9 

2004-05 

Cohort 1 




2005-06 

Cohort 2 

Cohort 1 



2006-07 

Cohort 3 

Cohort 2 

Cohort 1 


2007-08 

Cohort 4 

Cohort 3 

Cohort 2 

Cohort 1 


Note. Bold text denotes the current evaluation year. 


Table 2.5 shows that about three-fourths of seventh graders (Cohort 1) and sixth graders (Cohort 2) are 
economically disadvantaged. Comparison groups have nearly equal proportions of disadvantaged and 
minority students, and female and male students. The main difference between groups is the greater 
proportion of limited English proficient (LEP) students in treatment schools. 


Table 2.5. Demographic Characteristics of Students: 2005-06 



Cohort 1 (Grade 7) 

Cohort 2 (Grade 6) 


Immersion 

Control 

Immersion 

Control 

Characteristic 

N 

% 

N 

% 

N 

% 

N 

% 

Enrollment 

2,627 

— ] 

2.911 

— 

2,685 

— ] 

2,822 

— 

Economic disadvantage 

1,972 

75.1 

2.135 

73.4 

2,044 

76.1 

2,064 

73.3 

Ethnicity 









African American 

161 

6.1 

213 

7.3 

163 

6.1 

232 

8.2 

Hispanic 

1,900 

72.4 

1,948 

66.9 

1,960 

73.0 

1,900 

67.3 

White 

537 

20.4 

739 

25.4 

526 

19.6 

677 

24.0 

Other 

28 

1.1 

11 

0.4 

35 

1.3 

13 

0.5 

Limited English prof. 

611 

23.3 

460 

15.8 

811 

30.2 

523 

18.6 

Gender 









Female 

1,270 

48.3 

1,413 

48.5 

1,341 

49.9 

1,346 

47.7 

Male 

1,357 

51.7 

1,498 

51.5 

1,344 

50.1 

1,476 

52.3 


Source: Spring 2006 student database collected from 21 treatment and 21 control campuses. 
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Teachers 


During the 2005-06 school year, 1,257 teachers participated in the study (604 at treatment campuses 
and 653 at control campuses). Teachers in comparison groups are remarkably similar in terms of 
gender, ethnicity, advanced degrees, and average teaching experience. The number of teachers 
declined in the second year due to the exclusion of two campuses. 


Table 2.6. Demographic Characteristics of Teachers 



2004-05 

2005-06 

Characteristic 

Immersion 

N=22 

Control 

N=22 

Immersion 

N=21 

Control 

N=21 

Number of teachers 

622 

682 

604 

653 

% Female 

65.4 

68.8 

63.4 

68.3 

% Minority 

42.4 

35.3 

44.9 

43.3 

% African American 

7.8 

7.5 

2.8 

4.8 

% Hispanic 

32.2 

26.3 

40.4 

37.3 

% White 

57.6 

64.7 

55.1 

56.7 

% with no degree 

0.0 

2.0 

0.2 

0.3 

% with advanced degree 

21.7 

22.2 

21.2 

18.3 

Average years of teaching experience 

10.9 

11.4 

10.6 

11.5 


Note. The total number of teachers was 1,304 in 2004-05 and 1,257 in 2005-06 


Data Collection 

Data collection for the project began in August 2004. As Table 2.7 illustrates, researchers conducted 
site visits in each of the middle schools in fall 2004 and spring of 2005 and 2006. Additional 
measures, administered as pre-tests in fall and post-tests in spring, included teacher online surveys and 
student paper-and-pencil surveys. Additionally, we gathered school and student demographic, 
attendance, and achievement data from the Texas Public Education Information Management System 
(PEIMS) and Academic Excellence Indicator System (AEIS). In spring 2005 and 2006, individual 
middle schools submitted student-level data on disciplinary actions. 


Table 2.7. Time Frame for Data Collection: 2004-05 and 2005-06 



2004 

-05 

2001 

5-06 


Fall 

Spring 

Fall 

Spring 


2004 

2005 

2005 

2006 

Site visits (classroom observations ) 

X 

X 


X 

Teacher Survey (all teachers) 

X 

X 


X 

Teacher Survey (new teachers) 



X 


Student Survey (Cohort 1) 

X 

X 


X 

Student Survey (Cohort 2) 



X 

X 

Style of Learning Inventory (Cohort 1) 

X 

X 


X 

Style of Learning Inventory (Cohort 2) 



X 

X 

Texas Assessment of Academic Skills (TAKS ) 


X 


X 

Attendance 


X 


X 

Disciplinary actions 


X 


X 


Note. Data collection for 22 treatment and 22 control schools in 2004-05 and 21 treatment and 21 control schools in 2005-06. 
TAKS and attendance data also were collected for spring 2003 and spring 2004. 
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Measures 


Instruments measuring mediating and outcome variables included surveys and student performance 
measures. Survey items and scale scores reliabilities are provided in Appendix B. 

Teacher Survey 

Immersion and control teachers completed an online technology survey in fall 2004 (September to 
October) and spring 2005 (April to May). Additionally, in fall 2005 (September to October), teachers 
new to the schools completed the survey, and in spring 2006 (April to May), all teachers completed 
surveys. The survey included items related to school technology, teachers’ technology proficiency and 
use, and professional development experiences. In fall 2004, 1,271 teachers completed surveys (97% 
of all teachers, 97% of treatment, and 98% of control). In spring 2005, 1,144 teachers (88% of all 
teachers, 87% of treatment, and 88% of control) completed surveys. In spring 2006, 1,175 teachers 
completed surveys (93% of all teachers, 92% of treatment, and 95% of control). 

School mediating variables. Teachers responded to 33 items pertaining to their perceptions of school 
technology. They rated their strength of agreement with statements on a 5-point scale ranging from 1 
(, strongly disagree ) to 5 ( strongly agree). Item analysis using maximum likelihood factor analysis with 
Varimax rotation revealed five distinct factors, including Leadership (12 items). Classroom 
Technology Integration (4 items), Technical Support (5 items). Innovative Culture (4 items), and 
Parent and Community Support (2 items). Measures of internal consistency (Cronbach’s alpha) for 
school-level factors ranged from 0.66 to 0.94. 

Teacher mediating variables. Teacher surveys included measures of mediating variables, with items 
pertaining to teachers’ perceptions of Technology Proficiency (27 items), Professional Productivity 
(17 items). Student Classroom Activities (17 items), and Collaboration (11 items related to teacher 
interactions with colleagues). Additionally, confirmatory analysis of items adapted from the Levels of 
Technology Implementation (LoTi) Questionnaire (Moersch, 2001) showed reasonable fit indices for a 
model having Technology Integration (10 items), Learner-Centered Instruction (4 items), and 
Resistance to Integration (3 items). Cronbach’s alpha for scales ranged from 0.70 to 0.99. 

For Technology Proficiency items, teachers indicated their skill level on a 7-point scale with 1 and 2 
indicating low proficiency ( not true of me now), 3, 4, and 5 indicating moderate proficiency 
(, somewhat true of me now), and 6 and 7 indicating proficiency (very true of me now). Measures of 
integration — Technology Integration, Learner-Centered Instruction, and Resistance to Integration — 
also involved a 7-point scale ranging from 1 (not true of me now) to 7 (very true of me now). For 
Professional Productivity, Student Classroom Activities, and Collaboration, teachers used a 5-point 
scale to rate the frequency of activities or interactions: 1 (never), 2 (rarely-e.g., a few times a year), 3 
(sometimes-e.g., once or twice a month), 4 (often-e.g., once or twice a week), and 5 (almost daily). 

Student Surveys 

Cohort 1 students completed paper-and-pencil questionnaires measuring their technology proficiency 
and use in fall 2004 and spring 2005 as sixth graders, and as seventh graders in spring 2006. 
Additionally, they completed the Style of Learning Inventory (SLI) during the same time periods. The 
SLI, a paper-and-pencil questionnaire measuring student self-directed learning (i.e., self-generated 
behaviors oriented toward the attainment of learning goals), was administered as a baseline measure in 
fall 2004 and as a post-measure in spring of 2005 and 2006. Cohort 2 students (sixth graders) 
completed technology surveys in fall 2005 and spring 2006; they completed the SLI during the same 
two time periods. 
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Technology survey. Survey items measured students’ Technology Proficiency (22 items). Classroom 
Activities (12 items), Technical Problems (6 items), Small-Group Work (6 items), and School 
Satisfaction (6 items). Cronbach’s alpha coefficients ranged from 0.77 to 0.94. As a measure of 
Technology Proficiency, students indicated how well they could use various technology applications 
on a 5-point scale: 1 (I can do this not at all or barely), 2 (I can do this with some difficulty), 3 (I can 
do this fairly well), 4 {I can do this very well), and 5 {I can do this extremely well). For measures of 
Classroom Activities, Technical Problems, and Small-Group Work, students used a 5-point scale to 
rate the frequency of activities or interactions: 1 (never), 2 (. rarely-e.g ., a few times a year), 3 
(sometimes-e.g., once or twice a month), 4 (often— e.g., once or twice a week), and 5 ( almost daily). 
Students rated school satisfaction items on a 5-point agreement scale ranging from 1 ( strongly 
disagree) to 5 (strongly agree). 

Technology survey response rates for students are summarized in Table 2.7. Response rates were in 
the 80% to 90% range across time periods, and there were only slight differences in response rates 
between cohorts and comparison groups. 


Table 2.7. Student Technology Survey Response Rates: 2004-05 and 2005-06 



Fall 2004 

Spring 2005 

Fall 2005 

Spring 2006 


N 

% 

N 

% 

N 

% 

N 

% 

Cohort 1 









Treatment 

2,319 

90 

2,053 

80 

— 

— 

2,291 

87 

Control 

2,505 

84 

2,485 

83 

— 

— 

2,544 

87 

All 

4,824 

87 

4,538 

82 

— 

— 

4,835 

87 

Cohort 2 









Treatment 

— 

— 

— 

— 

2,209 

84 

2,379 

89 

Control 

— 

— 

— 

— 

2,405 

86 

2,452 

87 

All 

- 

- 

- 

- 

4,614 

85 

4,831 

88 


Style of Learning Inventory. The SLI is a 48-item survey, developed by the Metiri Group (2004), that 
is based on a model of self -regulated learning (Schunk & Zimmerman, 1998). The items on the SLI are 
categorized into 12 scales and three groupings. The three grouping and related scales are listed below. 

• Forethought is defined as influential processes and beliefs that precede efforts to learn (goal 
setting, strategic planning; self-efficacy beliefs; goal orientation; and intrinsic interest), 

• Performance/Volition control refers to processes that occur during learning efforts and affect 
concentration and performance (attention focusing, self-instruction, imagery; self-monitoring; 
and help seeking), and 

• Self-reflection involves processes that occur after learning efforts and influence a learner’ s 
reaction to that experience. Since the learning process is cyclical, these processes will in turn 
influence forethought regarding subsequent learning efforts (self evaluation, attributions, self 
reactions, and adaptivity). 

Students rated statements regarding their personal self-direction on a 7-point scale, ranging from 1 
(completely false) to 7 (completely true). A factor analysis of fall 2004 SLI data revealed low internal 
consistency of scales and groupings. Since no scales or groupings had sufficient reliability (a = 0.18 to 
0.52), analyses were limited to the SLI total score (a = 0.89). 

Table 2.8 summarizes SLI response rates. Responses ranged from 77% to 82% across time periods. 
With the exception of the spring 2005 SLI administration, there were only slight differences in 
response rates between cohorts or comparison groups. 
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Table 2.8. Style of Learning Inventory Response Rates: 2004-05 and 2005-06 



Fall 2004 

Spring 2005 

Fall 2005 

Spring 2006 


N 

% 

N 

% 

N 

% 

N 

% 

Cohort 1 









Treatment 

2,142 

83 

2,174 

85 

— 

— 

2,116 

80 

Control 

2,442 

82 

2,120 

71 

— 

— 

2,387 

81 

All 

4,584 

82 

4,294 

77 

— 

— 

4,503 

81 

Cohort 2 









Treatment 

— 

— 

— 

— 

2,115 

80 

2,198 

82 

Control 

— 

— 

— 

— 

2,265 

81 

2,228 

79 

All 

- 

- 

- 

- 

4,380 

80 

4,426 

80 


Observation of Teaching and Learning 

Researchers conducted fall 2004 classroom observations in a sample of Cohort 1, sixth-grade, core- 
subject classrooms (reading/English language aits, mathematics, science, and social studies). Spring 
2005 observations also focused on Cohort 1 students’ classrooms. In spring 2006, we expanded the 
sample to include seventh-grade teachers of Cohort 1 students and sixth-grade teachers of Cohort 2 
students. 

The Observation of Teaching and Learning (OTL) form documents basic descriptive information (e.g., 
number of students, content area), technology access and use (i.e., technology available and used by 
the teacher and students), and classroom environment (i.e., organization and management). In 
addition, researchers used time-interval ratings to record information in six areas: class organization 
(e.g., individual students, pairs, small groups, whole group), teacher activities (e.g., directing, guiding 
substantive discussion), teacher’s technology use (e.g., peripherals, presentation software), student 
activities (e.g., listening, learning facts, definitions, algorithms), students’ technology use (e.g., 
express themselves in writing, learn/practice skills), and student engagement (rated on a 5-point scale 
from low engagement to high engagement). 

Observers made the first rating after observing for 5 minutes, then made a rating every 10 minutes. 
During the observation, observers also recorded descriptive notes on the lesson objectives, teachers’ 
questioning strategies (lower or higher order), and class activities. Observations lasted about 45 
minutes. After the observation, and based on time-interval ratings and descriptive notes, observers 
rated the intellectual challenge of classroom work. Relying on rubrics developed by Newmann, 

Secada, and Wehlage (1995), observers rated four dimensions of intellectual challenge on a 5-point 
scale: Higher Order Thinking, Disciplined Inquiry, Substantive Conversation, and Value Beyond 
School. An aggregate score across three of the scales was used as an overall measure of the 
Intellectual Challenge. We excluded the Substantive Conversation scale because ratings were biased 
by classroom organization. Classes with teacher-directed instruction provided better opportunities to 
document the nature of substantive conversations. 

Number of observations. During fall 2004, researchers conducted observations at 22 middle schools 
(11 treatment and 1 1 control). In spring of 2005 and 2006, we expanded observations to all of the 
middle schools. In fall, researchers observed 128 classrooms (64 treatment and 64 control); in spring 
2005, we conducted follow-up observations, when possible, in the same classrooms. Altogether, we 
observed 240 classrooms (117 treatment and 123 control) in spring 2005. The following year (spring 
2006), we observed 243 classrooms (130 treatment and 1 13 control). These observations included a 
nearly equal mix of sixth- and seventh-grade classrooms. At small campuses, researchers observed 
nearly all core-subject teachers. For larger campuses, we observed at least eight classrooms (about half 
of core teachers). 
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Training procedures. Prior to site visits in fall 2004 and spring of 2005 and 2006, researchers 
participated in one- or two-day training events. Training activities informed data collectors about the 
research design, aspects of technology immersion, data collection protocols, effective interview and 
focus group techniques, and classroom observation procedures. Approximately half of each training 
event was devoted to the establishment of inter-rater agreement on the OTL form. During observation 
training, raters first reviewed background information and individual item and code definitions in the 
OTL manual. Raters next viewed a video in which a classroom teacher used technology as part of a 
lesson. The trainer stopped raters at 10-minute intervals to record ratings, discuss the extent of 
agreement or disagreement, and resolve misunderstandings. This process was repeated for an 
additional classroom video. 

To further enhance inter-rater agreement, raters were paired for observations in classrooms during 
visits to a middle school selected for training purposes. Following paired classroom observations in 
these schools, raters again discussed assigned ratings and resolved disagreements. Subsequently, for 
site visits to treatment and control middle schools, observers were paired for about 25% of classroom 
observations. Overlapping observations allowed the calculation of observer reliability (i.e., the 
percentage of agreement on ratings from paired observations). Additionally, paired observations 
supported the use of Many-facets Rasch Analysis (MFRA) to adjust scale scores on the Intellectual 
Challenge factor for the relative difficulty of each scale and the relative severity (or leniency) of each 
observer. 

Inter-rater agreement. Inter-rater agreement has been established for the Intellectual Challenge 
component of the classroom observation instrument. For this element, observers used 5 -point rating 
scales to measure four dimensions of a lessons’ intellectual demand (Fligher-Order Thinking. 
Disciplined Inquiry. Substantive Conversation, and Value Beyond School). Observer reliability on 
these scales was measured by calculating the percentage of time observers agreed on ratings from 
paired observations. Analyses of observations from fall 2004 indicated 78% inter-rater agreement. 
Agreement reached 98% when scale categories were allowed to vary by one scale point (on the 5-point 
scale). Inter-rater agreement declined somewhat in spring of 2005 and 2006. Exact agreement was 
63% and 62%, respectively, and 89% and 93% when ratings varied by one scale point. 

An overall measure of Intellectual Challenge for each teacher was constructed using Many-Facets 
Rasch Analysis (MFRA). The quality of instruction measure is an aggregate score across three scales 
(Fligher Order Thinking, Disciplined Inquiry, and Value Beyond School). The measure is adjusted for 
the relative difficulty of each scale and the relative severity (or leniency) of each observer. MFRA 
produces several fit statistics that can be used to measure each observer’s intrarater reliability or 
internal consistency. One of these, observer infit, weights each standardized residual by its variance 
and is more sensitive to unexpected patterns of small residuals. A second statistic, observer outfit, is 
an unweighted mean-square residual sensitive to outlying residuals (Linacre, 2004). There is no fixed 
rule for setting upper and lower limits for theses fit statistics. “Misfitting” raters have been defined as 
having either a mean-square infit or outfit statistic greater than 1.5 (Lunz, Wright, & Linacre, 1990), or 
the range has been from 0.5 to 3.0 (Myford & Wolfe, 2000). We define a “misfitting” observer as one 
with either a mean-square infit or outfit statistic less than 0.5 or greater than 1.5. This defines “misfit” 
as less than 50% of the variance in ratings than is modeled (a muted pattern) and more than 50% of the 
variance than is modeled (a noisy pattern). 

Observation data in fall 2004, spring 2005, and spring 2006, respectively, resulted in observer infit 
values from 0.61 to 1.34, 0.61 to 1.34, and 0.43 to 1.59, and observer outfit values from 0.62 to 1.20, 
0.62 to 1.20, and 0.40 to 1.67. While the spring 2006 fit statistics extended slightly beyond the 0.5 to 
1.5 range, mean infit and outfit values were in the 0.90 to 1.00 range. As a whole, no unusual rating 
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patterns appeared to be present in the classroom observation data, with only slightly unpredicted or 
overly predictable ratings (Linacre, 1995). 

Texas Assessment of Knowledge and Skills (TAKS) 

The TAKS is Texas’ criterion-referenced assessment that annually measures students’ mastery of the 
state’s content standards. TAKS assesses reading at grades 3 to 9; English language aits at grades 10 
and 11; writing at grades 4 and 7; mathematics at grades 3 to 11; science at grades 5, 8, 10, and 11; 
and social studies at grades 8, 10, and 1 1. Stringent quality control measures are applied at all stages 
of test administration, scanning, scoring, and reporting. Internal consistency reliabilities for TAKS 
assessments are in the high .80s to low .90s range. Evidence also supports the content, construct, and 
criterion-related validity of TAKS assessments. 2 

Cohort 1 students completed the TAKS reading and mathematics assessments as a pretest in 2004 (5th 
grade) and as posttests in 2005 (6th grade) and 2006 (7th grade). Cohort 1 students completed the 
TAKS writing assessment in 2003 (4th grade) and 2006 (7th grade). Cohort 2 students completed the 
TAKS reading and mathematics assessments as pretests in 2005 (5th grade) and as posttests in 2006 
(6th grade). 

At grades 6 and 7, TAKS reading measures four objectives: understanding of culturally diverse written 
texts, knowledge of literary elements, use of strategies to analyze written texts, and application of 
critic al-thinking skills. TAKS mathematics at grades 6 and 7 measures six objectives: numbers, 
operations, and quantitative reasoning; patterns, relationships, and algebraic reasoning; geometry and 
spatial reasoning; concepts and uses of measurement; probability and statistics; and mathematical 
processes and tools used in problem solving. At grade 7, TAKS writing measures six objectives: given 
a context, produce an effective composition for a specific puipose; demonstrate a command of 
conventions of spelling, capitalization, punctuation, grammar, usage, and sentence structure; recognize 
appropriate organization of ideas in written text; recognize correct and effective sentence construction 
in written text; recognize standard usage and appropriate word choice in written text; proofread for 
correct punctuation, capitalization, and spelling in a written text. 

School Attendance and Disciplinary Actions 

Post-measures of student attendance for Cohort 1 came from PEIMS data for the 2004-05 and 2005-06 
school years; attendance data for the previous two school years (2002-03 and 2003-04) served as pre- 
measures. Similarly, for Cohort 2, student attendance data for 2005-06 provided a post-measure while 
data for two school years (2003-04 and 2004-05) served as pre-measures. Additionally, individual 
campuses submitted data for student disciplinary actions taken during the 2005-06 school year. Data 
files included an indicator for the total number of Disciplinary Action Reports (PEIMS 425 records) 
reported for each student (Cohorts 1 and 2) during the school year. 


Technical information is available on the Texas Education Agency website at http://www.tea.state.tx.us/student 
assessment/resources/techdig04/index.html. 
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3. Technology Immersion — Second-Year Implementation 


Understanding whether or not an intervention has an effect on desired outcomes first requires an 
examination of the extent to which the intervention (in this case technology immersion) has been 
implemented as designed. Accordingly, this chapter begins with a description of technology immersion 
and the use of technology immersion packages as a means to operationally define the treatment and 
ensure more consistent measurement of implementation across sites. Next, we su mm arize research 
showing the important role of implementation in explaining project outcomes and describe our 
approach to measuring the level and quality of implementation of technology immersion. Finally, 
findings are presented on the fidelity of second-year - implementation at the 21 treatment campuses, 
with information derived from data collected during the 2005-06 school year. 

Defining Technology Immersion 

As a way to ensure consistent interpretation of technology immersion and comparability across sites, 
the Texas Education Agency (TEA) issued a Request for Qualifications (RFQ) that allowed 
commercial vendors to apply to become providers of technology immersion packages (TEA, 2003). 
Although state statute gives a general description of technology immersion, the concept and its 
component parts were defined operationally to foster continuity across treatment campuses. Vendor 
applicants to the RFQ had to include the following six components in their plan: 

• A wireless mobile computing device for each educator and student on an immersed campus 
to ensure on-demand access to technology; 

• Productivity, communication, and presentation software for use as learning tools; 

• Online instructional resources that support the state curriculum in English language arts, 
mathematics, science, and social studies; 

• Online assessment tools to diagnose students’ strengths and weaknesses or to assess their 
progress in mastery of the core curriculum; 

• Professional development for teachers to help them integrate technology into teaching, 
learning, and the curriculum; and 

• Initial and ongoing technical support for all parts of the package. 

Through a competitive application and expert review process, the TEA selected three lead vendors as 
providers of technology immersion packages (Dell Computer Inc., Apple Computer Inc., and Region 1 
Education Service Center [ESC]). Prices for technology immersion packages varied according to the 
numbers of students and teachers, the type of laptop computer, and the vendor provider. Package costs 
ranged from about $1,100 to $1,600 per student. Of the 21 immersion sites studied in the second year, 
5 middle schools selected the Apple package, 15 selected the Dell package, and 1 school selected the 
Region 1 ESC package (Dell computer). Table 3.1 provides an overview of the basic components 
within each package and the individual vendors that provided various products. 
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Table 3.1. Technology Immersion Packages 


Component 

Apple 

N= 5 Schools 

Dell 

N= 1 5 Schools 

Region 1 ESC 
N= 1 School 

Wireless laptop computer 

Apple 
iBook G4 

Dell Inspiron 
or Latitude 

Dell 

Inspiron 

Productivity software 

AppleWorks 

MS Office 
eChalk 

MS Office 
eChalk 

Online resources 

Various 

V arious 

Various 

Online assessment 

AssessmentMaster 

i-Know 

i-Know 

Professional development 

Apple Model 

Pearson Achievement 

Classroom Connect 

Technical and pedagogical 
support 

Apple, 

Campus/District 

Dell, 

Campus/District 

ESC 

Campus/District 


Wireless Laptops and Productivity Software 

All vendors offered a wireless laptop as the mobile computing device. Campuses could select either 
Apple laptops (iBook and MAC OSX) or Dell laptops (Inspiron or Latitude with Windows OS). For 
Apple laptops, AppleWorks provides a suite of productivity tools, including Keynote presentation 
software, Internet Explorer, Apple Mail, iCal calendars, iChat instant messaging, and iLife Digital 
Media Suite (iMovie, iPhoto, iTunes, Garage Band, and iDVD). For Dell laptops, Microsoft Office 
includes Word, Excel, Outlook, PowerPoint, and Access. In addition, eChalk serves as a “portal” to 
other web-based applications and resources included in the immersion package and a student-safe 
email solution. Region 1 ESC provided Dell products. 

Online Instructional and Assessment Resources 

Immersion packages included a variety of digital resources. Apple included the following online 
resources: netTrekker (an academic Internet search engine). Beyond Books from Apex Learning 
(reading, science, and social studies online), ClassTools Math from Apex Learning (complete math 
instruction), ExploreLearning Math and Science (supplemental math/science curriculum), 
TeenBizfOOO from Achieve 3000 (differentiated reading instruction), and My Access Writing from 
Vantage Learning (support for writing proficiency). Dell, Inc. selected netTrekker (an academic 
Internet search engine) and Connected Tech from Classroom Connect (technology-based lessons and 
projects). Region 1 ESC selected Connected Tech but also added a variety of teaching and learning 
resources including Unitedstreaming (digital videos), Encyclopedia Britcmnica, EBSCO (databases), 
NewsBank, and KI2 Teaching and Learning Center. For the Apple package, AssessmentMaster 
(Renaissance Learning) provides a formative assessment in all four core subject areas. Both the Dell 
and Region 1 ESC packages provide i-Know (CTB McGraw Hill) for core-subject assessment. In 
addition, all campuses have access to the online Texas Mathematics Diagnostic System (TMDS) that 
is provided free of charge by the state. 

Professional Development 

Each immersion package includes a different professional development provider. Apple uses its own 
professional development model, whereas the Dell package relies on Pearson Achievement Solutions, 
a commercial provider (formerly Co-nect ), to support professional development. Region 1 ESC uses a 
combination of service center support plus other services offered through Connected Coaching and 
Connected University. Although the professional development models and providers differ, they all 
were expected to include some common required elements, such as support for immersion package 
components, the design of technology-enhanced learning environments and experiences, lesson 
development in the core-subject areas, sustained learning opportunities, and ongoing coaching and 
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support. Individual districts and campuses collaborated with vendors to develop specific professional 
development plans for their teachers and other staff. 

Technical and Pedagogical Support 

Each technology immersion package provider also is required to provide campus-based technical 
support to advance the effective use of technology for teaching and learning. Apple designed a Master 
Service and Support Program. Dell established a Call Center dedicated to technical support for TIP 
grantees as well as an 800 telephone number for hardware and software support. Region 1 ESC had an 
online and telephone HelpDesk to answer questions and provide assistance. 

In sum, the RFQ process created technology immersion packages with common elements. Still, the 
complexity and variability of the treatment makes it critically important for researchers to document 
not only how and how well technology immersion is implemented but also to identify factors that 
contribute to implementation variations. 

Association of Implementation and Outcomes 

The study of large-scale educational reforms and organizational change has shown that the level and 
quality of implementation largely determines the achievement of desired outcomes (Berman & 
McLaughlin, 1978; Borman, Hewes, Overman, & Brown, 2003; Datnow, Borman, & Stringfield, 

2000; Fullan & Stieglbauer, 1991). Studies of technology innovations, likewise, reveal that ineffective 
implementation undermines prospects for changes in student learning opportunities and academic 
outcomes (Cuban, 2001; Cuban, Kirkpatrick, & Peck, 2001). Lessons learned from earlier studies of 
school reform led Desimone to conclude: “. . .it is first necessary to measure the degree of 
implementation before assessing outcomes and attempting to attribute them to a specific program” 
(2002, p. 437). 

Building on prior experience and research, the focus of school improvement has shifted recently from 
program-centered reforms to initiatives that focus on changing the whole school as a way to foster 
improved student outcomes through changes in teaching and learning. Increasing numbers of schools, 
particularly low performing ones, have undertaken comprehensive school reform. Spurred by the 
availability of federal funds and the No Child Left Behind Act of 2001, hundreds of comprehensive 
school reform (CSR) models have been developed and implemented in schools. As a result, the nature 
of model implementation and the school change process have been studied widely. Although 
technology immersion and CSR models are not identical, they share the common focus of changing 
the whole school, including changes to the curriculum and delivery of instruction. Thus, past research 
contributes to a clearer understanding of current implementation findings. 

Evidence from a meta-analysis of CSR achievement effects shows that implementation counts. Using 
the best available measure of implementation — the number of years a CSR model was implemented in 
a school — researchers found an increasing effect on achievement outcomes associated with a greater 
number of years of implementation. The CSR effect size was relatively strong in the first year (0.17), 
but there was a tendency for new initiatives to weaken in the second, third, and fourth years. On the 
other hand, schools that implemented models for five or more years showed achievement advantages 
nearly twice as large (Borman et al., 2003; Borman, 2005). These long-term effects, however, may be 
influenced by the large proportion of schools (up to one third) that discontinued use of CSR models 
within the first few years, and consequently, were excluded from longitudinal analyses (Borman, 
2005). 


23 



More recent, in-depth studies of CSR model implementation demonstrate that achieving quality 
implementation is challenging. Studies involving hundreds of CSR schools revealed that none of the 
schools had fully implemented all components of the models they had adopted. Schools appeared to 
implement components selectively (Kurki, Aladjen, & Carter, 2005). And notably, schools had more 
difficulty following instructional practices prescribed by their model and practices aimed at increasing 
parental involvement in school affairs (Vernez, Karam, Mariano, & Dc Mart ini, 2006). In these studies, 
higher levels of implementation were associated with higher levels of support (e.g., principal 
leadership, teachers’ commitment, model developer support, professional development). 

Findings regarding the influence of contextual variables (school size or student characteristics) were 
mixed as were results regarding the improvement of implementation over time. Kurki, Aladjen, and 
Carter (2005) cited increased implementation for most indicators between the first and third year, 
whereas Vernez et al. (2006) found that the first year or so, for the most part, determined the degree of 
implementation, with levels remaining fairly constant across three years. It remains to be determined 
whether implementation as measured in these studies links to student academic outcomes. Still, 
findings as a whole, point to the crucial need to measure the extent of implementation before drawing 
conclusions about the effectiveness of technology immersion. 

Measuring Implementation Fidelity 

Measurement of second-year implementation of technology immersion builds upon research 
conducted during the first project year. Direction for the refinement of the technology immersion 
model and its measurement came from first-year reports describing the nature of project 
implementation and factors that promoted or impeded success, 1 reviews of immersion packages, and 
discussions with project and vendor staffs. As in the first year, implementation is measured as the 
fidelity with which technology immersion components and related elements attain an envisioned 
“ideal.” This approach involves gathering extensive data on immersion components at each of the 
treatment campuses and comparing campus-to-campus variations with the vision for “full” 
implementation. 

In contrast to the first year, we adopted a two-part approach in the second year. First, we used 
indicators to describe each campus’ progress on a 4-step scale toward immersion standards. Rating 
scales for components and related elements identify four levels of immersion: minimal (0 to 1.99), 
partial (2.00 to 2.99), substantial (3.00 to 3.49), and full (3.50 to 4.00). Second, we used quantitative 
implementation indices to gauge the level of technology immersion using standardized scores 
(z scores). Both the immersion standard scores and implementation indices are derived from values for 
seven components: (a) Leadership, (b) Teacher Support, (c) Parent and Community Support, 

(d) Technical Support, (e) Professional Development, (f) Classroom Immersion, and (g) Student 
Access and Use. Scores for components and their elements come from surveys of teachers ( At 5 60. 
including 318 core-subject teachers) and students (N-7,022) at 21 treatment schools. 

Table 3.2 provides descriptions of the technology immersion indicators that contributed to both the 
immersion standard scores and implementation indices. Appendix C provides additional technical 
detail on the measurement of implementation fidelity and scoring rubrics that describe the four levels 
of immersion for immersion components and their related elements. 


1 Shapley et al. (2006a). Evaluation of the Texas Technology Immersion Pilot: First-Year Results. Austin, TX: Texas Center 
for Educational Research. Shapley et al. (2006b). Evaluation of the Texas Technology Immersion Pilot: An Analysis of 
Baseline Conditions and First-Year Implementation of Technology Immersion in Middle Schools. Austin, TX: Texas Center 
for Educational Research. 
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Table 3.2. Description of Implementation Indicators for Technology Immersion 

Leadership 

To what extent do teachers indicate that the principal establishes a clear vision and expectations, encourages 
integration, provides supports, and involves staff in making decisions about instructional technology. 

Teacher Support 

To what extent do teachers share an understanding about technology use, do teachers continually learn and 
seek new ideas, are teachers unafraid to learn about and use technologies, and are teachers supportive of 
integration efforts. 

Parent and Community Support 

To what extent do teachers believe that parents and the surrounding community support the school’s efforts 
with technology. 

Technical Support 

To what extent do teachers indicate that technical problems with computers, Internet access, repairs, and 
material availability pose barriers to technology immersion. 

Professional Development 

Contact Hours: To what extent does the duration (hours) of technology-related professional development 
(PD) support the integration of technology into teaching, learning, and the curriculum. 

Classroom Support: To what extent do core-subject teachers receive coaching or mentoring from an internal 
source, such as another teacher or technology coordinator, or an external (non-school) source. 

Content Focus: To what extent do core-subject teachers indicate that PD emphasizes curriculum, instructional 
methods, and lesson development in core subjects. 

Coherence: To what extent do core-subject teachers indicate that PD is consistent with personal and school 
goals, builds on prior learning, and supports state standards and assessments. 

Classroom Immersion 

Technology Integration: To what extent do core teachers alter instructional practices, allocate time, integrate 
research on teaching and learning, improve basic skills, and support higher order thinking through technology. 
Learner-Centered Instruction: To what extent do teachers have students establish learning goals, use 
information and inquiry skills, complete alternative assessments, and have active and relevant learning 
experiences. 

Student Classroom Activities: To what extent do teachers have students use particular technology resources 
for learning in core-subject classes, such as a word processor for writing, a spreadsheet for calculation or 
graphing, or the Internet for research. 

Communication: To what extent do teachers use technology to communicate with students, parents, and 
colleagues or to post information on a class website. 

Professional Productivity: To what extent do teachers use technology to enhance their professional 
productivity (e.g., keep records, analyze data, develop lessons, deliver information). 

Student Access and Use 

Laptop Access: To what extent do students have access to wireless laptops throughout the school year. 

Core-Subject Learning: How frequently do students use technology resources for learning in core-subject 
classes. 

Home Learning: To what extent do students have access to and use laptops outside of the school for 
homework and learning. 

Note. See Appendix C for a technical description of the measurement of implementation indicators. 

Computing Implementation Scores 
Scores for Immersion Standards 

We used teacher and student survey data to compute implementation scores for indicators that 
measured progress toward immersion standards (i.e., minimal to full implementation). Adapting a 
process developed by the RAND Corporation, 2 the value for each indicator was computed relative to 
the maximum value (4.00 — the value assigned to full implementation). Standardization based on the 
maximum value allowed comparisons across different types of indicators. For each component and 
element of technology immersion, standardization involved the following computations: 


2 

“ Vernez, G., Karam, R., Mariano, L.T., & DeMartini, C. (2006). Evaluating Comprehensive School Reform Models at Scale: 
Focus on Implementation. Santa Monica, CA: RAND. 
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• Agreement scales (i.e., strongly agree or strongly disagree with a prescribed practice or 
behavior): 4 = strongly agree, 3 = agree, 2 = neither agree nor disagree, 1 = disagree, and 
0 = strongly disagree. 

• Frequency scales (i.e., four- or five-level frequencies of doing a prescribed practice): 

4 = highest frequency met, 3 or 2.67 = second highest frequency, 2 or 1.33 = third-highest 
frequency, 1 = fourth-highest frequency, and 0 = never or do not do. 

• Continuous variables (i.e., how much time or how often a prescribed practice is done): 

4 = meet or exceed requirements, and 0-3.99 = proportional fraction of requirement. 

Scores for Implementation Indices 

In addition to the standards-based scoring system described above, we used teacher and student survey 
data to compute standardized implementation indicators (z scores with a mean of 0 and standard 
deviation of 1.0) that could then be aggregated to generate: 

• A single implementation score for each technology immersion component for each school 
(e.g.. Leadership Index), and 

• an overall implementation score for each school (Implementation Index). 3 


Implementation of Technology Immersion 

The sections to follow present findings on (a) the extent to which schools provided the implementation 
supports considered essential to advance technology immersion, and (b) the degree to which schools 
and their teachers implemented components, including those relevant to classroom immersion 
practices and students’ technology access and use. We first present campus-level results for the 
Implementation Index (z score), which provides an overall measure of technology immersion, and then 
use implementation standards (measured at four levels) to describe the level of implementation for the 
model’s core components. 

Implementation Index 

Campus-level results for the Implementation Index displayed in Figure 3.1 illustrate the wide variation 
in the levels of technology immersion for the 21 middle schools in the second project year. The 
Implementation Index, which measures the overall presence of the seven components of immersion, is 
a z score with a mean of 0 and a standard deviation of 1.0. Thus, the score for each campus indicates 
how many standard deviations from the mean a score lies. Campuses with scores above 0 have higher 
values on the components of technology immersion, whereas campuses with index values below zero 
show less evidence of the immersion components. Results suggest that about a third of middle schools 
(6), with Implementation Index scores ranging from 0.97 to 1.91 standard deviations above the mean, 
have a much stronger presence of the components of technology immersion compared to other 
schools, thus a higher level of implementation that more nearly approximates expected standards. 


3 Variables were standardized as z scores from their original scale or continuous variable values. The use of z scores rather 
than the immersion standard scores was necessary in order to aggregate data across variables that had widely varying 
standard deviations. 


26 




Figure 3.1. Campus means for 21 immersion middle schools (MS) on the Technology Immersion 
Implementation Index (standardized scores [z scores] with a mean of 0 and a standard deviation of 1.0). 


Although the Implementation Index is helpful in comparing the relative level of implementation across 
campuses, it is also important to examine the degree to which middle schools are attaining the 
standards that represent what a substantially or fully immersed campus should achieve. As another 
way to gauge successful implementation, we used standards-based scores that describe the extent to 
which treatment schools received various supports for implementation and the degree to which schools 
implemented the instructional and learning components of immersion as designed. 


Implementation Standards 


As explained previously, progress toward technology immersion standards is measured at four levels 
0 minimal , 0-1.99; partial , 2.00-2.99; substantial, 3.00-3.49; and full immersion, 3.50-4.00) across 
seven components. Five components assess the strength of supports for technology immersion 
(Leadership, Teacher Support, Parent/Community Support, Technical Support, Professional 
Development), whereas two components gauge the extent of technology immersion (Classroom 
Immersion and Student Access and Use). Figure 3.2 displays the mean implementation scores by 
component. Mean immersion standard scores ranging from 2.48 to 3.06 showed that supports for 
technology immersion from school principals, teachers, the community, technical staff, and 
professional development providers did not meet full implementation standards (mean score of 3.50 to 
4.00). With insufficient levels of support, classroom- and student-related components of immersion 
were implemented to a modest extent. Teachers, on average, reported only partial levels of Classroom 
Immersion (M = 2.48) and students, as a whole, reported partial (but even lower) levels of technology 
access and use (M = 2. 17). Results for individual components are discussed in detail below. 
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Figure 3.2. Mean level of implementation (measured on a 0 to 4 scale) for seven Technology Immersion 
components (N=21 middle schools). 4 

Level of Principal, Teacher, and Parent/Community Support 

The technology immersion model calls for the systemic integration of technology across key areas of 
the school, including teaching and learning, educator development, administration and support 
services, and infrastructure for technology. Momentum for implementation, thus, depends upon the 
backing and support of individuals, establishment of institutional norms, and assistance from the 
surrounding community. Figure 3.3 shows teachers’ reported support for implementation from key 
constituents. 



Figure 3.3. Level of implementation (measured on a 0 to 4 scale) for Technology Immersion components 
(Leadership, Teacher Support, Parent and Community Support) by the mean implementation score and 
percentage of schools at each implementation level. 


4 Standards-based scores for Professional Development, Classroom Immersion, and Student Access and Use are averages 
across elements of these components. These scores serve descriptive purposes. Composite z scores are used in statistical 
analyses. 


28 



Leadership. Principals play a key role not only in the decision to become an immersion campus but 
also as the champion for school innovation and change, manager of reform efforts, and provider of 
needed resources. Accordingly, teachers at each school were asked to rate the extent to which their 
principals provided leadership for immersion. Principals demonstrated leadership through behaviors 
such as involving staff in decisions, setting clear expectations for technology use, encouraging and 
participating in professional development events, and providing resources and support. Results in 
Figure 3.3 show that teachers in about half of schools reported only partial levels of principal support 
for technology immersion (. M = 2.64). An additional half of campuses reported substantial levels of 
support (M = 3.48, indicating that they either agreed or strongly agreed that their principals provided 
technology-related leadership). In none of the schools did responding teachers strongly agree that 
principals provided the kind of leadership necessary for full immersion. 

Teacher Support. Ample evidence demonstrates that teachers are central to the implementation of 
any initiative aimed at fundamental school change. Specifically, teachers’ commitment to the reform is 
affected by their beliefs about the need for school reform and changes in classroom practice 
(Desimone, 2002; Fullan & Flargreaves, 1996; Hargreaves, 1994). Teacher buy-in is critically 
important for technology immersion because teachers are to a large extent the mediators of students’ 
technology experiences. Thus, it is noteworthy that these teachers reported modest support for 
technology. Teachers at a third of campuses reported partial levels of support (M — 2.78). That is, 
teachers at these schools were unsure that they shared an understanding about technology use for 
student learning, were continually learning and seeking new ideas, were not afraid to learn about and 
use new technologies, and were supportive of integration efforts. Although teachers at two-thirds of 
schools reported substantial support for innovative practices, their mean score (3.20) showed moderate 
rather than strong buy-in. 

Parent and Community Support. Since parents must share responsibility for an expensive laptop 
computer with their child or children, their understanding of and support for technology immersion is 
a key paid of implementation. Additionally, the enthusiastic support of community members, including 
elected members of the local school board and business people, may influence the level of 
implementation through mechanisms such as the adoption of supportive policies, provision of 
resources, or promotion of positive public relations. More important, a lack of parent and community 
support has been associated with the failure of other school reforms (Desimone, 2002). Unfortunately, 
at most of the 21 middle schools, the level of parent and community support was less than expected. 
Teachers at three-fourths of campuses reported minimal (M = 1.59) or partial (M = 2.53) levels of 
parent and community support. On the other hand, teachers at a quarter of campuses reported 
substantial support (M = 3.13), with teachers generally agreeing that parents and the surrounding 
community supported their efforts with technology. 

Level of Technical and Pedagogical Support 

Technical and pedagogical supports arc critical aspects of the technology immersion model. As 
schools build their network infrastructure and acquire computer hardware and technology resources, 
ongoing technical support for all components of immersion and ongoing professional development in 
integrating technology into teaching and learning arc essential for successful implementation. 

Technical Support. Technical support for immersion is expected to be provided by the vendors who 
furnish technology immersion packages as well as district- and campus-level staff who assist with 
implementation and offer timely support when technical problems arise. As with the other support 
mechanisms described above, the level of technical support for immersion typically failed to meet 
expectations (see Figure 3.4). Teachers at about four-fifths of campuses reported a partial level of 
technical support for implementation (M = 2.62). These teachers were generally unsure that school 
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computers are kept in working order, requests for assistance are addressed in a timely way, Internet 
connections work adequately, and classroom materials are readily available. Teachers at a few 
campuses reported a substantial level of technical support (M = 3.12), and teachers at one campus 
reported a full level of support (M = 3.50). Findings as a whole suggested that inadequate levels of 
technical support for technology immersion challenged most middle school teachers. 



□ Partial □ Substantial DFull 


Figure 3.4. Level of implementation (measured on a 0 to 4 scale) for Technical Support component by mean 
implementation score and percentage of schools at each implementation level. 

Professional Development. Each of the technology immersion packages was required to include a 
professional development component designed to support technology immersion for all educators on 
an implementing campus. The technology immersion model calls for professional development that 
instructs educators in effective classroom integration and is delivered through proven methods (i.e., 
learning through a variety of delivery systems, collaboration, sustained learning opportunities, and 
ongoing coaching and support). Findings for elements of professional development displayed in 
Figure 3.5 show that schools’ staffs typically were not exposed to either the prescribed amount or type 
of professional development. 

Although professional development providers were obligated to support all teachers, we concentrated 
on core-subject teachers because of their close association with measurable student outcomes. First, 
core teachers at the majority of campuses reported receiving less than the prescribed number of hours 
of technology-related professional development over the first two implementation years (estimated by 
project staff and vendors to be about 80 hours). Teachers in about half of schools reported a minimal 
level of professional development (M = 1.28, 35 or less hours, on average). In contrast, teachers in 
more than a third of schools received nearly the requisite number of hours (substantial to full 
implementation levels). Additionally, core teachers typically reported that they received only a 
minimal (M = 1.76) or partial (M = 2.30) level of classroom support for technology immersion. This 
meant that teachers as a whole rarely (a few times a year) or never received classroom coaching or 
mentoring from an internal source (such as another teacher or technology coordinator) or external 
source (such as a vendor-provided professional trainer). 
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Figure 3.5. Level of Implementation (measured on a 0 to 4 scale) for elements of the Professional 
Development component (Contact Hours, Classroom Support, Content Focus, and Coherence) by mean 
implementation score and percentage of schools at each level. 

Core teachers who participated in technology-related professional development expressed varying 
views on the extent to which activities supported the curricular and instructional goals of the 
technology immersion model. Teachers at nearly half of schools reported that the content of 
professional development placed a minor emphasis on curriculum, instructional methods, and lesson 
development in core areas (M = 2.62, partial implementation). On the other hand, teachers at more 
than half of schools indicated there was at least a moderate to major emphasis on the prescribed 
content. 

Teachers also typically failed to see the coherence of technology-related professional development 
with their personal goals, earlier learning experiences, and state/district curriculum standards and 
assessments. Teachers at almost all campus reported that professional development was either not at 
all coherent ( M = 1.90, minimal implementation) or coherent to a minimal extent (M - 2.49, partial 
implementation). Teachers at only a few campuses reported that professional development was 
coherent to a moderate extent (M = 3.11, substantial implementation) or great extent (M - 3.53, full 
implementation). In general, it appeared that many core-content teachers did not receive the kind of 
professional development intended to advance technology immersion, and the nature of professional 
development received varied widely across schools. 

Level of Classroom Immersion 

Given the needed equipment, digital resources, and support for technology immersion, teachers are 
expected to design technology-enhanced learning environments and integrate technology into 
teaching, learning, and the curriculum. Ideally, a technology immersed classroom provides a means for 
more engaged, relevant, meaningful, and personalized student learning. Figure 3.6 illustrates teachers’ 
level of implementation relative to five elements of classroom immersion: Technology Integration, 
Learner-Centered Instruction, Student Classroom Activities (with technology), Communication, and 
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Professional Productivity. Overall, teachers reported only minimal to partial levels of implementation 
for the elements of classroom immersion. 


Technology Integration and Learner-Centered Instruction. Just as teachers as a whole must buy-in 
to the concept of technology immersion, individual core-subject area teachers must believe that new 
technologies are relevant to their particular curriculum and instructional practices and that new 
methods have the potential to positively influence student outcomes. Researchers have found that 
when the pedagogical design of reform models is more compatible with teachers’ own ideology, they 
embrace change more strongly (Datnow & Castellano, 2000). For the technology immersion model, 
teachers reported the strength of their affiliation with technology integration and learner-centered 
instruction. Notably, teachers at nearly all of campuses reported only minimal (M = 1.86) or partial (M 
= 2.53) levels of affinity for technology integration. Teachers at these schools reported it was either 
not true or just somewhat true that they now modify their instructional practices in significant ways 
through technology. In contrast, teachers at only two schools indicated that it is somewhat to very true 
now that they alter their instructional practices, allocate time, integrate current research on teaching 
and learning, improve basic skills, and support higher-order thinking through technology. (M= 3.06, 
substantial immersion). 
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Figure 3.6 Level of Implementation (measured on a 0 to 4 scale) for five elements of the Classroom 
Immersion component (Technology Integration, Learner- Centered Instruction, Student Activities, 
Communication, and Professional Productivity) by the mean implementation score and percentage of 
schools at each implementation level. 


Teachers’ responses relative to learner-centered instruction mirror their beliefs about technology 
integration. Teachers at almost all schools reported minimal (M - 1.77) or partial (M = 2.46) levels of 
implementation relative to the adoption of learner-centered practices. Core teachers at these campuses 
indicated that it was either not true or just somewhat true that their students establish learning goals, 
use information and inquiry skills, complete alternative assessments, and have active and relevant 
learning experiences. Teachers at just a tenth of campuses reported a substantial level of 
implementation (M = 3.05), saying that it was somewhat to very true that they now use learner- 
centered methods. Overall results indicated that core-subject teachers at many schools are not strongly 
committed to the instructional and learning practices advocated as paid of technology immersion, and 
in fact, these methods may be incompatible with many teachers’ ideological beliefs and values. 
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Student Classroom Activities. Teachers in immersed classrooms also arc expected to have their 
students use technology resources to support core-content learning on an almost daily basis. At about a 
fifth of schools, however, teachers of core subjects (English/language arts, mathematics, science, and 
social studies) reported that their students rarely (a few times a year) or never use technology 
resources (M = 1.89, minimal implementation). For example, students seldom use a word processor to 
write a story, use software to learn and practice skills, create a presentation, or conduct Internet 
research. At another four-fifths of schools, teachers reported that their students sometimes (once or 
twice a month) use various technology resources to support core-content learning (M = 2.44, partial 
implementation). On average, teachers at none of the campuses reported substantial to full 
implementation, with students using technology resources often (once or twice a week) to almost 
daily. Findings indicated that teachers, in general, are not using fully the instructional and learning 
resources provided as paid of the technology immersion packages. 

Communication and Professional Productivity. Wireless computing devices and supporting 
software provided through technology immersion packages allow educators and students to use 
technology on a daily basis as a tool for learning, communication, and productivity. Appleworks 
provided a suite of communication tools at the Apple campuses, whereas at Dell campuses, Microsoft 
Office and eChaik provided software and a web-based portal for resources and email. Despite the 
availability of communication tools, teachers at a quarter of campuses reported a minimal level of 
implementation for communication (M= 1.63). Teachers at these campuses rarely (a few times a year) 
or never used technology to communicate with students, parents, or their colleagues, or to post 
information for students on a class website. Teachers at an additional half of schools reported that they 
just sometimes (once or twice a month) used technology for communication (M = 2.50, partial 
immersion). On the other hand, teachers at a few campuses reported a substantial level of 
implementation (M = 3.15), with email and web-based communication tools used often (once or twice 
a week). 

In contrast to other elements of classroom immersion, teachers more often reported the use of 
technology to enhance their professional productivity. Teachers at three-quarters of campuses 
indicated that they sometimes used technology for activities such as keeping records, analyzing data, 
developing lessons, and delivering information using presentations. Teachers at an additional quarter 
of campuses reported a substantial level of implementation, with technology used often (once or twice 
a week) for professional productivity purposes. Overall findings on teachers’ reported levels of 
classroom immersion are generally consistent with other research on teachers’ adoption of technology- 
based practices. In the early stages of implementation, teachers discover the potential of technology 
use for increased productivity and begin to use it as an instructional tool on a limited basis (Sandholtz, 
Ringstaff, & Dwyer, 1997). 

Level of Student Technology Access and Use 

Although the transformation of teaching and learning in the classroom is a critical part of technology 
immersion, the model also aims to provide on-demand technology access for students, which allows 
them to become more independent and self-determined learners both within and outside of the school. 
In a fully immersed school, all students should have access to their wireless laptops and resources 
nearly the entire school year (about 170 to 180 days). Student access to their laptops, however, may be 
affected by a number of factors, such as disciplinary infractions, technical issues, time for repairs, and 
parent resistance. Overall data reported by students indicates that the number of days that students 
actually had laptops available for use out of the 1 80-day school year varied widely both across and 
within schools (see Figure 3.7). At about a sixth of campuses, students had a minimal level of access 
(M= 1.40), indicating that laptop access days varied to an extremely large extent (from 42 to 169 days 
per student). Students at about half of campuses had only a partial level of access (M = 2.57), with 
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student access days varying to a large extent (from 100 to 176 days per student). At about a third of 
campuses, students reported a substantial level of access (M = 3.15), signifying that student access 
varied from about 140 to 178 days per student. Conversely, students at about a tenth of campuses had 
a full level of access, with laptops available the targeted 170 to 180 days per student. 


3.86 
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Figure 3.7. Level of Implementation (measured on a 0 to 4 scale) for elements of the Student Access and 
Use component (Laptop Access Days, Core- Content Learning, and Home Learning) by mean 
implementation score and percentage of schools at each implementation level. 

Students also estimated how often they used their laptops in their English/language arts, mathematics, 
science, and social studies classes. Similar to teachers, students at more than one-third of campuses 
reported a minimal level of implementation (M = 1.59), indicating that they rarely (a few times a year) 
or never used technology-resources in core-subject classes. Students at the remaining two-thirds of 
schools reported a partial level of implementation (M = 2.37), with laptops used sometimes (once or 
twice a month) to often (once or twice a week) in core classrooms. While students as a whole used 
their laptops infrequently for learning in core classrooms, they use them even less often for learning 
outside of school. Students in almost two-thirds of schools reported a minimal level of laptop use for 
home learning (M= 1.40). These students used their laptops outside of school for homework and 
learning either not at all or to a trivial extent. Students in the remaining third of schools used their 
laptops for home learning at a minimal level (M - 2.25). Students at none of the schools used their 
laptops outside of school to either a substantial or full level of immersion (i.e., used laptops to a 
moderate or large extent, respectively). 

Overall, students’ opportunities to use their laptops both within classrooms and outside of school were 
affected by the number of days that students actually had their laptops. As noted earlier, in some 
schools, students’ laptop access days were drastically reduced by factors such as disciplinary actions 
and time for repairs. Students in other schools, contrary to the tenets of technology immersion, were 
not allowed to take their laptops home or their home use was restricted in some way (e.g., laptops 
could only be used for special assignments). 

In sum, overall results for the implementation of technology immersion suggest that in the second 
year, the level of implementation varied widely by campus and none of the middle schools achieved 
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full immersion. Results for the campus Implementation Index combined with evidence from standards- 
based scores suggest that about a third of campuses (6) reached substantial immersion, whereas the 
majority of campuses (15) achieved only minimal to partial immersion in the second year. 

Implementation Indices by Component of Technology Immersion 

To further illustrate each school’s level of immersion, Table 3.3 presents the composite campus 
Implementation Index (z score) alongside implementation indices (z scores) for each of the seven 
components. Despite some variations in component scores, middle schools that had higher values on 
the Implementation Index tended to have component scores that indicated a stronger presence of the 
immersion attributes such as principal leadership and teacher support for immersion. In contrast, 
middle schools that had extremely negative values on the Implementation Index generally had 
negative values for nearly all of the immersion components. These findings suggest that the 
implementation indices are relatively effective in discriminating higher and lower implementing 
schools. Still, there are noteworthy exceptions to the prevailing trends. Some schools, such as MS 21, 
had generally lower implementation values for most of the indicators except Classroom Immersion 
(0.84). This suggests that teachers at that school are making strides toward classroom integration of 
technology even though strong supports are not in place. In contrast, despite strong values for support 
indicators, teachers in MS 1 have a relatively low presence of classroom immersion attributes. The 
relationships among implementation variables are explored further in the section to follow. 


Table 3.3. Second-Year Implementation of Technology Immersion 


Middle 

School 

(MS) 

Leader- 

ship 

Index 

Teacher 

Support 

Index 

Parent/ 

Comm. 

Index 

Technical 

Support 

Index 

PD 

Index 

Classroom 

Immersion 

Index 

Student 

Access/Use 

Index 

Implemen- 

tation 

Index 

MS 1 

1.78 

1.18 

1.05 

1.28 

2.09 

0.58 

1.78 

1.91 

MS 2 

0.97 

1.15 

1.60 

1.45 

0.86 

0.58 

0.91 

1.48 

MS 3 

0.65 

1.25 

0.50 

-0.39 

1.52 

1.49 

1.12 

1.20 

MS 4 

0.79 

1.24 

1.45 

0.32 

0.78 

0.28 

0.62 

1.07 

MS 5 

1.13 

0.29 

0.61 

2.67 

-0.04 

0.07 

0.21 

0.97 

MS 6 

1.12 

0.78 

1.07 

0.73 

0.53 

0.32 

0.38 

0.97 

MS 7 

0.36 

0.42 

0.95 

-0.54 

0.26 

1.68 

0.10 

0.64 

MS 8 

0.43 

0.44 

0.50 

-0.74 

1.91 

-1.03 

0.53 

0.40 

MS 9 

0.58 

0.20 

0.03 

-0.80 

0.03 

1.06 

-0.59 

0.10 

MS 10 

-0.60 

-0.16 

0.12 

-0.31 

-0.18 

0.95 

-0.05 

-0.04 

MS 11 

0.16 

0.22 

-0.14 

0.24 

-0.98 

0.72 

-0.48 

-0.05 

MS 12 

-0.89 

-0.35 

-0.72 

-0.75 

-0.16 

0.53 

0.90 

-0.28 

MS 13 

-0.62 

0.33 

-0.79 

1.37 

-1.44 

-1.74 

0.70 

-0.43 

MS 14 

-0.69 

-0.28 

-0.55 

0.08 

-0.83 

-0.86 

0.23 

-0.57 

MS 15 

-0.46 

-0.33 

0.05 

-0.45 

-1.13 

-1.31 

0.63 

-0.59 

MS 16 

0.92 

0.11 

-0.14 

-1.11 

-0.23 

-1.27 

-2.55 

-0.84 

MS 17 

-0.19 

-0.72 

-0.04 

-1.42 

-0.53 

-0.39 

-1.16 

-0.87 

MS 18 

-0.77 

-0.12 

-1.16 

-0.36 

-0.07 

-0.57 

-1.68 

-0.93 

MS 19 

-1.03 

-1.99 

-0.73 

-0.45 

-0.19 

-1.07 

-0.46 

-1.16 

MS 20 

-1.42 

-1.03 

-1.10 

-0.15 

-1.30 

-0.85 

-0.63 

-1.27 

MS 21 

-2.20 

-2.65 

-2.59 

-0.68 

-0.92 

0.84 

-0.50 

-1.70 


Note. Implementation indices are z scores with a mean of 0 and a standard deviation of 1 .0. Scores above zero indicate a 
greater presence of technology immersion components and higher levels of implementation. 
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Factors Associated with Implementation 


In this section, we explore relationships among various implementation components of technology 
immersion and examine whether particular support mechanisms or school characteristics are 
associated with teachers’ self-reported levels of classroom immersion and students’ estimations of 
their technology access and use. The strength of relationships between implementation levels for 
immersion components (mean campus z scores) and school characteristics (mean enrollment counts, 
percentages of minority and economically disadvantaged students, and percentages of students passing 
TAKS tests) arc examined individually through bivariate correlations. 

Immersion Components 

Table 3.4 displays the correlations between the seven components of technology immersion, with 
statistically significant coefficients denoted in bold. As anticipated, teachers’ perceptions of their 
principal’ s leadership for immersion was strongly associated with their collective support for 
technology innovation, as well as their views on parent and community support, the presence of 
technical support, and the robustness of professional development. And reasonably, teachers’ 
commitment to technology innovation was significantly related to other support mechanisms in 
addition to the level of students’ technology access and use. On the contrary, teachers’ support for 
technology innovation showed almost no association with the strength of their classroom immersion. 

Table 3.4. Correlations of Technology Immersion Components 

Components of Technology Immersion 

Leader- Teacher Parent Technical Class Student 

Components ship Support Support Support PD Immersion Access/Use 

Leadership 

Teacher Support .85*** 

Parent/Community Support .88*** .84*** 

Technical Support ,42f .42t .38t 

Professional Development .69** .61** .65** 

Classroom Immersion .26 .02 .30 

Student Access and Use .29 .45* .46* 

Note, t p < .10, *p < .05, **p < .01, ***p < .001. PD = Professional Development. 

Technical support for immersion was most strongly associated with students’ reported access to and 
use of laptops, suggesting that stronger technical support reduced laptop maintenance problems that 
kept computers out of the hands of students. The intensity of campus professional development 
supporting immersion was significantly associated with higher levels of implementation for both 
classroom immersion and student access and use components. Significant correlations also were seen 
between the strength of professional development and teachers’ perceived levels of support from 
principals, technical leaders, and parent and community members. 

The campus level of classroom immersion, unexpectedly, was significantly associated with only one 
implementation component (professional development). On the other hand, the ultimate goal of 
technology immersion — students’ technology access and use for learning — was significantly related to 
increased implementation levels for four of five support components. Principals’ leadership appeared 
to affect students indirectly through the facilitation of other supports for immersion. 

School Characteristics 

We also explored the relationship between implementation components and school characteristics (see 
Table 3.5). Middle-school campus characteristics included the average student enrollment, percentage 


.13 

.02 ,38t 

.49* .45* .28 
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of minority students (African American and Hispanic), school poverty (percentage of economically 
disadvantaged students as measured by eligibility for federal free- and reduced-price lunches), and 
achievement (percentage of grades 6 to 8 students passing all TAKS tests in spring 2006). 


Table 3.5. Correlations of Technology Immersion Components and School Characteristics 



Characteristics of Middle Schools j 

Immersion Components 

Student 

Enrollment 

Percent 

Minority 

Students 

Percent 

Economically 

Disadvantaged 

Students 

Percent of 
Students 
Passing All 
TAKS Tests 

Leadership 

-.04 

-.23 

-.30 

.14 

Teacher Support 

-.03 

-.29 

-.42f 

.29 

Parent & Community Support 

.06 

-.19 

-.40f 

.18 

Technical Support 

-.46* 

-.12 

-.22 

.32 

Professional Development 

-.05 

-.01 

-.20 

.13 

Classroom Immersion 

.14 

-.08 

-.16 

-.22 

Student Access & Use 

-.24 

-.17 

-.411 

.23 

Implementation Index 

-.12 

-.21 

-,42f 

.21 


Note, t p < .10, *p < .05, **p < .01, ***P < 001. 


Results showed that school size was negatively associated with implementation levels for most 
technology immersion components. That is, schools with larger student enrollments tended to have 
slightly lower levels of implementation than schools with fewer students, although the negative 
relationship was significant for only one component. Teachers at larger schools reported significantly 
lower levels of technical support, which indicated that technical problems at these schools posed a 
greater barrier to technology immersion. 

Higher percentages of minority students (African American and Hispanic) also showed a weakly 
negative relationship with implementation components, whereas higher percentages of economically 
disadvantaged students at a school surfaced as having an overall negative and sometimes significantly 
negative relationship with implementation levels. Schools with more disadvantaged populations had 
significantly lower levels of implementation for teacher support, parent and community support, 
student technology access and use, and for the composite Implementation Index. 

In contrast to the negative relationships between school demographic characteristics and levels of 
implementation, the school’s achievement context was positively associated with nearly all of the 
implementation indicators, although correlations were generally low. Teachers’ reported level of 
classroom immersion was the only immersion component that was negatively correlated with 
achievement. 

Classroom Immersion 

To further understand teachers’ perspectives, an additional analysis examined the relationships among 
support components, school characteristics, and elements of core-subject teachers’ classroom 
immersion. Correlation coefficients presented in Table 3.6 showed generally low associations among 
variables, with some positive and some negative relationships. Still, a few statistically significant 
findings surfaced. Classroom immersion elements that assessed the strength of teachers’ ideological 
agreement with technology innovation and constructivist practices (technology integration and learner- 
centered instruction) were significantly related to teachers’ perceptions of the viability of various 
support components, including principal leadership, parent and community support, and professional 
development. Teachers’ perceptions of the robustness of professional development also showed the 
strongest relationship with the composite Classroom Immersion Index. 
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Table 3.6. Correlations of Support Components and School Characteristics by Elements of 
Classroom Immersion 


Indicators/Characteristics 


Core-Subject Teachers' 

1 Classroom Immersion 


Technology 

Integration 

Learner- 

Centered 

Instruction 

Student 

Activities 

Communi- 

cation 

Professional 

Productivity 

Classroom 

Immersion 

Index 

Leadership 

.44* 

.40* 

.09 

.10 

.02 

.26 

Teacher Support 

.35 

.33 

.02 

.23 

.04 

.26 

Parent & Community Support 

.46* 

.40f 

.09 

.22 

-.01 

.30 

Technical Support 

.09 

-.01 

-.02 

.11 

-.25 

.02 

Professional Development 

•38f 

.51* 

.23 

.16 

.27 

•38f 

School enrollment 

.05 

.08 

.09 

.17 

.25 

.14 

% minority students 

-.20 

-.10 

.12 

-.22 

.23 

-.08 

% Disadvantaged students 

-.17 

-.07 

.19 

-.43f 

• 11 

-.16 

% Students pass all TAKS 

-.24 

-.28 

-.47* 

.19 

-.39 

-.22 


Note, t p < .10, *p < .05, **p < .01, *** P < .001. 


On the other hand, a school’s mean achievement on TAKS was negatively associated with teachers’ 
reported implementation levels for five of six classroom immersion elements. Notably, the association 
between campus TAKS scores and teachers’ reported frequency of their students’ classroom 
technology activities was negative and statistically significant. This suggests that simply increasing the 
frequency of classroom technology use may not produce desirable student outcomes. The only positive 
association between classroom immersion indicators and student achievement was the frequency of 
teachers’ technology use for communication via email or a class website. Teachers at campuses with 
higher achieving students used technology more often for communication, whereas teachers at 
campuses with larger minority and disadvantaged student populations communicated with technology 
less often. 

Student Access and Use 

Correlations for students’ reported levels of technology access and use also showed important trends 
(Table 3.7). First, the strength of the composite Student Access and Use Index was significantly 
related to stronger levels of implementation for various support mechanisms (teacher, 
parent/community, and technical support, and teacher professional development). The Student Access 
and Use Index, however, was negatively associated with larger school size and higher percentages of 
minority and economically disadvantaged students. 


Table 3.7. Correlations for Support Components and School Characteristics 
by Elements of Student Access and Use 


Indicators/Characteristics 

Student Technology Access and Use 1 

Laptop 
Access Days 

Classroom 

Learning 

Home 

Learning 

Student 

Access/Use 

Index 

Leadership 

-.06 

.18 

.32 

.29 

Teacher Support 

.09 

.20 

.50* 

.45* 

Parent & Community Support 

.12 

.22 

.47* 

.45* 

Technical Support 

.45* 

.46* 

.18 

.49* 

Professional Development 

.04 

.28 

.46* 

.45* 

School Enrollment 

-.29 

-.59** 

.31 

-.24 

% Minority Students 

-.10 

-.29 

.05 

-.17 

% Disadvantaged students 

-.15 

-.31 

-.30 

-.41+ 

% Students pass all TAKS 

.05 

.20 

.17 

.23 


Note, t p< -10, *p < .05, **p <.01, *** p < 001. 
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Second, as might be expected, the strength of campus technical support was significantly related to the 
number of days that students had their laptops available for use and the frequency with which they 
used their laptops in core-subject classrooms for learning. Results also suggested that certain types of 
support enabled students to use their laptops more productively at home. Significant correlations 
showed that the level of teacher and parent/community support for immersion, and the campus 
emphasis on professional development were positively associated with the extent to which students’ 
reported using their laptops for learning at home (i.e., homework and learning games). Finally, 
findings showed that the robustness of a student’s technology access and use was associated with the 
characteristics of the school that he or she attended. Students attending larger schools and schools with 
larger minority and economically disadvantaged populations reported generally lower levels of 
technology access and use. Given the variations in student experiences across immersion campuses, it 
was noteworthy that students’ reported technology access and use was positively, though weakly, 
associated with campus academic achievement. 

Conclusions 

This chapter described technology immersion components, as defined by the Texas Education Agency 
and further operationalized through technology immersion packages. In the second year, we measured 
implementation using a two-part approach: (a) designation of standards defining four levels of 
immersion ( minimal , partial, substantial, and full), and (b) calculation of standardized implementation 
indices (z scores). Both types of scores arc derived from values for components relative to supports for 
immersion, and the extent of classroom immersion and students’ technology access and use. Major 
findings are the following: 

• The majority of middle schools struggled in the second year to implement the prescribed 
components of technology immersion. The Implementation Index, a composite score 
measuring the overall presence of immersion components, showed that about a third of middle 
schools had a much stronger level of immersion than the other schools (i.e., substantial 
immersion). No middle schools reached full immersion in the second year. 

• Many schools needed stronger supports in the areas of school leadership, teachers’ 
commitment to innovative technology practices, parents’ support for students’ technology use, 
and technical supports that addressed obstacles to technology use . 5 

• Core-subject teachers, as a whole, are using new technology resources to enhance their own 
professional productivity and to support existing instructional practices. Teachers at many 
schools, however, use technology resources sporadically in their classrooms. While some 
teachers are making noteworthy progress toward classroom immersion, other teachers believe 
new technologies are less relevant to their curriculum and instructional practices and, in some 
instances, inconsistent with their ideological beliefs and values. 

• Students’ access to and use of laptops for learning within and outside of school fell short of 
expectations. Some schools allowed students to have unlimited home access, while others 
restricted out-of-school laptop use. Restrictions diminished the possibilities for student 
learning with laptops at home. Likewise, teachers’ varying commitment to classroom 
immersion affected students’ opportunities to learn with technology in core-content classes. 
Overall, students’ experiences with laptops varied substantially across classrooms and schools. 


While there were substantial differences in available supports among immersion schools, immersion teachers as a 
whole perceived significantly stronger school-level supports for technology than control teachers. See comparisons 
between immersion and control teachers in Appendix D. 
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Despite low levels of implementation at many campuses, report chapters to follow demonstrate that 
technology immersion can positively affect teachers and students in many ways even at lower 
implementation levels. 
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4. Effects of Technology Immersion on Teachers and Teaching 


In the theoretical model, researchers hypothesize that given quality implementation of technology 
immersion (i.e., supportive school leaders, robust student technology access, effective use of online 
curricular and assessment resources, professional development supporting curricular integration, and 
adequate technical and pedagogical support to maintain an immersed campus), one might expect 
school-level improvements in teachers’ commitment to technology innovation, technical support for 
implementation, and parent and community support. Effective principal leaders are believed to provide 
the kinds of supports that fortify implementation. A more supportive school environment should lead 
in turn to teachers who have greater technology proficiency, use technology more for their own 
professional productivity, hold a more favorable pedagogical orientation toward technology, and 
collaborate more often with their peers to advance teaching and learning through technology. 
Moreover, teachers in schools that achieve a higher level of school and classroom immersion will use 
laptops as a tool to increase the intellectual challenge of lessons and will have students who use 
technology more in their classrooms. 

Contrary to expectations regarding the implementation of technology immersion, results reported in 
Chapter 3 revealed wide valuation from school-to-school in the levels of implementation. None of the 
21 middle schools had reached full immersion in the second project year, and in fact, results showed 
that just about a third of middle schools approached a substantial level of immersion. More important, 
two-thirds of schools achieved only minimal to partial implementation. Recognizing that 
implementation levels at most schools in the second year failed to meet specified standards, we 
investigated whether or not there is an immersion effect on teachers given that the level of 
implementation is generally low. 

Findings on the effects of technology immersion on teacher-mediating variables that underpin the 
theoretical model come from online surveys of teachers completed in fall 2004 (September to October) 
and again in spring of 2005 and 2006 (April to May). Teachers responded to items pertaining to their 
personal and classroom technology experiences. In fall 2004, 1,271 teachers completed the survey 
(97% of treatment and 98% of control teachers), and in spring 2005, 1,144 teachers (87% of treatment 
and 88% of control) responded to the survey. In spring 2006, 1,175 teachers completed the online 
survey (92% of treatment and 95% of control). Additionally, researchers conducted classroom 
observations during site visits to each of the treatment and control schools to gather information on 
instructional practices and changes across time. We conducted observations in a sample of sixth-grade, 
core-subject classrooms in fall 2004 (64 treatment and 64 control teachers) and again in spring 2005 
(117 treatment and 123 control teachers). In spring 2006, researchers conducted observations in both 
sixth- and seventh-grade classrooms (130 treatment and 113 control teachers). 

Teacher Mediating Variables — HLM Analysis 

Surveys included measures of seven teacher-level valuables. Teachers responded to items gauging their 
technology knowledge and skills (Technology Proficiency and Professional Productivity); the strength 
of their ideological views relative to Technology Integration, Learner-Centered Instruction, and 
Resistance to Integration; the frequency of Student Classroom Activities with technology; and their 
propensity for Collaboration with peers on technology. Cronbach’s alpha reliability coefficients for the 
teacher-level scale scores ranged from 0.70 to 0.99. (See Appendix B for technical details.) 
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One advantage of a longitudinal study is the potential to study the nature of teacher change. The 
development of hierarchical linear models (HLM) has provided statistical tools for studying rates of 
change using measurements from multiple time points (Raudenbush & Bryk, 2002). For this study, we 
measured teacher variables on three occasions: fall 2004 (baseline), spring 2005 (after the first 
implementation year) and spring 2006 (after the second implementation year). The analyses that 
follow contrast immersion and control teachers’ individual growth trajectories for each of the seven 
scales described above. We analyzed effects using three-level hierarchical growth models. FILM 
growth models produce teacher- and school-specific effects (i.e., the extent which the survey scores 
vary across time, teachers, and schools). In our models, we hypothesize that school poverty is related 
to teachers’ initial status and yearly growth rate. This supposition stems from an investigation of the 
implementation of technology immersion indicating that a higher concentration of economically 
disadvantaged students in a school is negatively associated with stronger levels of school and 
classroom immersion. Similarly, other research reviews confirm negative effects of school poverty on 
school reform efforts (Desimone, 2002) and student achievement (Sirin, 2005). Since TIP grants 
targeted high-needs schools, the percentages of disadvantaged students are generally high across most 
of the study’s schools. Even so, poverty concentrations vary substantially within comparison groups 
(ranging from 34.5% to 99.6% for treatment campuses and from 33.7% to 100% for control 
campuses). The statistical model is described below. 

Level 1 : Repeated-Measures Model 

Level 1 is a repeated-measures model (i.e., survey time within teachers) that enables us to capture key 
features of growth (e.g., initial status, rate of change). In the model, Y nj is the survey scale score at year 
t for teacher / in school j. Survey Time is the point at which teachers completed the online surveys 
(0=fall 2004, I =spring 2005, 2=spring 2006). The key parameters in the model are noij and nuj. The 
coefficient n 0 y represents the “initial status” (that is, the initial survey scale score) for teacher i in 
school j in fall 2004, and Knj is the growth rate (rate of change) for teacher i in school j per school 
year. The e tij is the error term (within-teacher measurement error) assumed to be normally distributed 
with a mean of 0 and a constant variance. Thus, at level 1 the model is 

Y,ij - n 0 ij + n nj (Survey Time) tij + e tij . 

Level 2: Teacher-Level Model 

The Level 2 model (between-teachers model) allows us to determine differences between teachers in 
features of growth (e.g., initial status, rate of change). In the teacher-level model, noij is the teacher’s 
initial survey scale score and n U j is the teacher’s rate of growth per school year. In the model, (S 0 oj 
represents the mean initial status within school j, and p IO j is the mean yearly rate of teacher change 
within school j. The r 0 y and r /y - are residuals (i.e., random effects). At level 2, the model is 

7Z0ij — ftoOj + fOij. 

Klij=PlOj + t'lij. 

Level 3: School-Level Model 

At the school level (level 3), we examined how teachers’ initial status and growth varied across 
schools as a function of school- level random effects ( ju 0 oj and n 10] ) as well as school conditions, 
including immersion status and school poverty. That is, we hypothesized that being in an immersion 
school is positively related to teachers’ growth on technology-related scores, after controlling for the 
poverty level of the school. Thus, we pose the following school-level model: 

fiooj= yooo + Yooif Immersion Status)j + yoo 2 (School Poverty )j + fJooj- 

fiioj= yioo + yioi(Immersion Status)j + yw 2 (School Poverty), + /i 10 j. 
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In the model, fiooj is the mean initial status for teachers in school j and y 0 oo is the overall mean 
initial status (grand mean); p 10j is the mean teacher growth rate in school j and y 100 is the overall 
mean teacher growth rate. Immersion status is an indicator variable with a value of 0 for a control 
school and a value of 1 for an immersion school. School poverty is a continuous variable with 
percentages ranging from 33.7% to 100%. The coefficients y 001 and y m represent the direction and 
strength of association of immersion status and school-level initial status. 

Effects of Immersion on Teachers 

Analyses of immersion effects included 802 teachers, with 364 in immersion schools and 438 in 
control schools. The results presented in Table 4.1 show that after adjusting for school poverty, 
technology immersion had a statistically significant effect on teachers’ rates of growth for six 
technology-related variables. Teachers at technology immersion schools, on average, had significantly 
steeper growth trends than teachers at control schools for Technology Proficiency and Professional 
Productivity, two measures of teachers’ ideology (Technology Integration and Learner-Centered 
Instruction), and the frequency of Student Classroom Activities (with technology) and Collaborative 
interactions with colleagues on technology-related issues. In contrast, for the Resistance to Integration 
scale, there was no significant effect of immersion on teachers’ mean rate of change. 

Table 4.1. Immersion Effects on Estimated Mean Growth Rates for Teacher Variables 


Statistics for Teachers in Immersion Schools 
with Average School Poverty c 



Immersion 
Effect Net of 
School 
Poverty 

Average 
Estimated 
Initial Status 
Fall 2004 

Yearly 
Growth Rate 

Average 

Estimated 

Score 

Spring 2006 

Yearly 
Growth Rate 
for Control 
Teachers 

Technology Proficiency 11 

Yes 

4.33 

0.35*** 

5.03 

0.18 

Professional Productivity 13 

Yes 

2.93 

0.26*** 

3.44 

0.09 

Ideology 






Technology Integration 11 

Yes 

3.07 

q yg*** 

4.66 

0.27 

Learner-Centered Instruction 11 

Yes 

3.60 

0.48*** 

4.55 

0.20 

Resistance to Integration 11 

No 

2.12 

-0.01 

2.10 

0.08 

Student Classroom Activities' 3 

Yes 

1.92 

0.33*** 

2.57 

0.04 

Collaboration 13 

Yes 

2.39 

0.21** 

2.81 

0.06 


Source: Online teacher surveys conducted in fall 2004, spring 2005, and spring 2006. Note. *p < .05; **p < .01; ***p < .001. 
‘‘Items measured on a 7-point scale. b Items measured on a 5-point scale. 


Figure 4.1 illustrates the estimated growth trajectories for immersion teachers in schools with average 
levels of school poverty. Results show that these teachers have positive growth trajectories for all of 
the technology-related indicators, with the exception of Resistance to Integration, which remains 
stable across years. There were notable school poverty effects for both immersion and control 
schools. Teachers working on campuses with higher concentrations of school poverty grew, on 
average, at a significantly slower rate than their counterparts in more affluent schools on indicators for 
Technology Proficiency and Technology Integration, and at a somewhat slower rate on indicators for 
Professional Productivity and Learner-Centered Instruction. Tables 4.2, 4.3, and 4.4 provide school- 
level statistics for the HLM analyses of immersion effects on teacher technology-related variables. 
Sections to follow explain results related to changes in teachers’ knowledge and skills, ideology, 
classroom practices, and peer collaboration. 
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Figure 4.1. Estimated mean growth trajectories for Technology Immersion teachers working in 
schools with average levels of school poverty on technology-related indicators (ratings on 
either 5-point or 7-point scales). 

Technology Knowledge and Skills 

Texas Technology Applications Standards require all teachers to master and use technology-related 
terminology, concepts, and strategies, and to be able to use appropriate tools to accomplish a range of 
tasks (e.g., communicate with diverse audiences and analyze electronic information). Given the 
importance the state places on technology knowledge and skills and the potential impact of immersion, 
our online surveys included measures of teachers’ Technology Proficiency and Professional 
Productivity. 

For Technology Proficiency, teachers rated their skills in using various technology applications on a 7- 
point scale: 1 and 2 ( not true of me now)', 3, 4, and 5 ( somewhat true of me now)', and 6 and 7 ( very 
true of me now). The proficiency scale included items measuring technology operations (e.g., send 
email to coworkers, parents, or peers; search for and find a Web site; find primary sources of 
information on the Internet). Teachers also reported ratings on items related to classroom instruction, 
such as using the computer for presentations or creating a lesson plan or unit incorporating technology. 

Results in Table 4.2 and Figure 4.2 show that immersion teachers grew in technology proficiency at a 
significantly faster rate (0.35 scale-score points per year) than control teachers (0.18 points) between 
fall 2004 and spring 2006. Even though immersion teachers began with slightly lower mean 
proficiency scores than control teachers in fall, they surpassed control teachers in spring 2005 and 
continued to grow in proficiency during the next school year. 
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Table 4.2. Immersion (Fixed) Effect Analyses for Teacher Technology 
Knowledge and Skills Variables 


Dependent variable 
and predictor 

Technology Proficiency 11 
Gamma 

Coefficient f-value 

Professional 

Gamma 

Coefficient 

Productivity 

f-value 

Initial status (fall 2004) 

4.476 

51.88*** 

2.984 

53.30*** 

Immersion 

-0.151 

-1.06 

-0.057 

-0.72 

School Poverty 

- 0.001 

-0.15 

0.002 

0.68 

Growth rate 

0.176 

5.26*** 

0.089 

4 49 *** 

Immersion 

0.178 

4 Q2*** 

0.167 

5.90*** 

School Poverty 

-0.004 

- 2 . 66 * 

- 0.001 

- 1.11 


'p < .10; *p < .05; **p < .01; ***p < .001. 


Teachers who taught at immersion and control schools with higher levels of school poverty 
(percentages of economically disadvantaged students) had significantly slower rates of growth for 
Technology Proficiency. For each percentage point increase in school poverty, teachers had a 0.004 
scale-score decrease in Technology Proficiency. Thus, as Figure 4.2 illustrates, a 20% increase in 
school poverty predicts a 0.08 point decrease in teachers’ yearly growth in proficiency (i.e., 20 x 
0.004). As the level of school poverty increases, the teacher proficiency gap continues to widen. 



— ♦ — Immersion: Lower School Poverty — ■ — Immersion: Higher School Poverty 
- - A- - Control: Lower School Poverty - - -X- - Control: Higher School Poverty 


Figure 4.2. Estimated mean growth trajectories for teachers at immersion and control schools 
for Technology Proficiency (ratings on a 7-point scale). Comparisons are for teachers working 
in schools with lower concentrations of school poverty, and teachers working in schools with a 
20 percentage point higher level of school poverty. 
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Teachers also rated the frequency with which they used technology for Professional Productivity on a 
5-point scale ranging from 1 (never) to 5 ( almost daily). Productivity items, for example, measured 
teachers’ use of technology for administrative, classroom management, communication, and 
instructional puiposes. Similar to findings for Technology Proficiency, teachers at immersion schools 
had significantly steeper rates of growth than control teachers in the use of technology to improve their 
productivity. The estimated yearly mean growth trajectories for immersion and control teachers in 
more advantaged schools were 0.26 and 0.09 scale-score points per year, respectively. Teachers 
working in schools with higher percentages of disadvantaged students had slightly slower growth 
rates. 

Ideology 

Teachers also responded to items measuring their ideological views relative to technology integration 
and constructivist practices on a 7-point scale, ranging from 1 (not true of me now) to 7 (very true of 
me now). Confirmatory factor analysis of these items, which were adapted from the Levels of 
Technology Implementation (LoTi) Questionnaire (Moersch, 2001), showed reasonable fit indices for 
a model having Technology Integration, Learner-Centered Instruction, and Resistance to Integration as 
latent variables. Results detailed in Table 4.3 show that teachers at immersion schools, on average, 
became more positive toward innovative technology practices across time. 


Table 4.3. Immersion (Fixed) Effect Analyses for Teacher Ideology Variables 


Dependent variable 
and predictor 

Technology 

Integration 11 

Learner-Centered 

Instruction 

Resistance 
to Integration 11 

Gamma 

Coefficient 

t- value 

Gamma 

Coefficient 

t- value 

Gamma 

Coefficient 

t- value 

Initial status (fall 2004) 

2.836 

41.80*** 

3.561 

69.45*** 

2.359 

43.59*** 

Immersion 

0.234 

2 . 02 * 

0.034 

0.37 

-0.240 

-3.20** 

School Poverty 

0.009 

2.83** 

0.005 

2.30* 

0.001 

0.76 

Growth rate 

0.265 

6.08*** 

0.200 

5.39*** 

0.083 

2 . 20 * 

Immersion 

0.528 

7 10*** 

0.278 

4 20 *** 

-0.092 

- 1.66 

School Poverty 

-0.004 

-1.74f 

- 0.002 

-1.18 

0.000 

-0.08 


< .10; *p < .05; **p < .01; ***p < .001. 

d Technology immersion teachers had significantly higher initial Technology Integration scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, showed that original differences were 
statistically insignificant (the difference divided by the standard error of the difference = 0.43). 
b Technology immersion teachers had significantly lower initial Resistance to Integration scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, showed that original differences were 
insignificant (the difference divided by the standard error of the difference = 0.42). 

The Technology Integration scale included items gauging teachers’ actions supporting curricular and 
instructional infusion of technology. For example, teachers indicated the extent to which computer- 
related activities enabled them to support students’ authentic problem solving or to promote critical 
thinking. Findings show that teachers in immersion and control schools initially expressed similar 
views about technology integration, but immersion teachers had a significantly more positive rate of 
change. The mean estimated growth trajectory for immersion teachers who worked in schools having 
average levels of school poverty was 0.79 scale points per year compared to 0.27 for control teachers. 
Teachers in schools with higher concentrations of school poverty had significantly slower rates of 
growth relative to technology integration practices. 

Teachers at immersion schools compared to control also changed at a significantly faster rate in their 
affiliations with principles of Learner-Centered Instruction. Across survey administrations, immersion 
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teachers reported increasingly higher ratings for items describing pedagogical practices such as having 
students establish individual learning goals, emphasizing experiential learning, and providing real- 
world experiences. The estimated yearly growth in the adoption of learner-centered practices for 
immersion and control teachers in more advantaged schools was 0.48 and 0.20 scale-score points, 
respectively. 

For the Resistance to Integration scale, teachers expressed their strength of association with items 
indicating that classroom computers are not a priority, not a necessary part of instruction, and not 
practical for students. Contrary to the two ideological indicators discussed above, there was little 
change in the growth rate on the Resistance to Integration scale for either immersion or control 
teachers. Scores indicated that teachers, on average, expressed a relatively low level of resistance to 
technology integration, and their level of resistance remained fairly stable across years. The extent of 
school poverty had a minimal effect on teacher resistance. 

Student Classroom Activities and Teacher Collaboration 

Table 4.4 provides statistics for scales measuring teachers’ classroom activities and collegial 
collaboration. This Student Classroom Activities scale provided an estimate of the frequency — on a 5- 
point scale ranging from 1 (never) to 5 ( almost daily ) — with which teachers had students in their 
typical class use technology in various ways. For example, teachers might have their students use 
technology for writing, learning and practicing skills, communication, or Internet research. As 
expected, given the availability of laptops at immersion schools, teachers at treatment schools had a 
significantly faster growth rate for Student Classroom Activities (0.33 and 0.04 scale-score points per 
year, respectively, for immersion and control teachers in lower poverty schools). School poverty had 
no discernable effect on teachers’ growth rate for the frequency of students’ classroom activities 
involving technology. Despite immersion teachers’ positive growth trends for the frequency of their 
students’ use of technology in the classroom, estimated mean scores indicated that by spring 2006, 
immersion teachers, on average, had students use various technology applications in their class 
infrequently (about once or twice a month, M = 2.57). 


Table 4.4. Immersion (Fixed) Effect Analyses for Student Classroom 
Activities and Teacher Collaboration Variables 


Dependent variable 
and predictor 

Student Classroom 
Activities 

Teacher 

Collaboration 11 

Gamma 

Coefficient 

1- value 

Gamma 

Coefficient 

f- value 

Initial status (fall 2004) 

1.864 

41.03*** 

2.255 

42.24*** 

Immersion 

0.052 

0.80 

0.138 

1.73 1 

School Poverty 

0.004 

2.07* 

0.003 

1.28 

Growth rate 

0.043 

2.01* 

0.055 

2.47* 

Immersion 

0.285 

8.03*** 

0.154 

3.72** 

School Poverty 

0.000 

0.13 

0.001 

1.14 


f p < .10; *p < .05; **p < .01; ***p < .001. 

d Technology immersion teachers had significantly higher initial Collaboration scores. A 
latent variable regression, controlling for the effect of this initial difference on the growth 
rate, showed that original differences were statistically insignificant (the difference 
divided by the standard error of the difference = 1.54 for Collaboration). 

We also reasoned that a greater abundance of technology resources and opportunities for shared 
professional development would lead to stronger teacher connections. Accordingly, the Collaboration 
scale measured teacher interactions with colleagues that supported improvements in instructional 
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practices, such coaching and mentoring, collectively developing technology lessons, and exchanging 
information about their students. As expected, immersion teachers had a steeper mean yearly growth 
trend for Collaboration (0.21 scale-score points) than control teachers (0.06 points). However, the 
growth rate coefficients for both teacher groups were significant, indicating that teachers across 
comparison groups were increasingly interacting with peers on technology practices. This may reflect 
the fact that both treatment and control campuses received Title II, Paid D grants, with 25% of dollars 
allocated for technology-related professional development. Campus poverty had a negligible effect on 
teacher collaboration. 


Effects of Immersion on Classroom Practice 

Researchers conducted classroom observations in a sample of sixth-grade core-subject classrooms in 
fall 2004 and spring 2005 (reading/English language aits, mathematics, science, and social studies 
teachers of Cohort 1 students). In spring 2006, we modified the classroom sample to include 
observations for seventh-grade teachers of Cohort 1 students and sixth-grade teachers of Cohort 2 
students. Classroom observations were conducted by single observers (about 75% of classrooms) and 
pairs of observers (about 25% of classrooms). Paired observations permitted the calculation of inter- 
observer agreement. In fall 2004, researchers observed 128 classrooms (64 treatment and 64 control) 
in 22 schools. In spring 2005 and 2006, we expanded observations to include all schools. We observed 
240 classrooms in 2005 (1 17 treatment and 123 control), and 243 classrooms in 2006 (130 treatment 
and 1 13 control). At small campuses, researchers observed nearly all core-content teachers; at large 
campuses, we observed at least eight classrooms (about half of core teachers). 

During observations, data collectors recorded descriptive information about the classroom 
environment; time-interval ratings of classroom organization, teacher activities and technology use, 
student activities and technology use, student engagement, and student collaboration. Observers also 
recorded notes during the observations to capture the lesson’s content focus and objectives, teachers’ 
questioning strategies (lower and higher order), and students’ learning experiences. 

Following classroom observations, observers used time-interval ratings and descriptive notes to rate 
the Intellectual Challenge of classroom work (rating scales developed by Newmann, Secada, & 
Wehlage, 1995). One section of the Observation of Teaching and Learning (OTL) instrument included 
5-point rating scales for four dimensions of the intellectual quality of instruction: 

• Construction of Knowledge: Higher Order Thinking. Instruction involves students in 
manipulating information about ideas by synthesizing, generalizing, explaining, 
hypothesizing, or arriving at conclusions that produce new meaning and understanding for 
them. 

• Disciplined Inquiry: Deep Knowledge. Instruction addresses central ideas of a topic or 
discipline with enough thoroughness to explore connections and relationships and to produce 
relatively complex understandings. 

• Disciplined Inquiry: Substantive Conversation. Students engage in extended conversational 
exchanges with the teacher or peers about subject matter in a way that builds an improved and 
shared understanding of ideas or topics. 

• Value Beyond School: Connections to the World Beyond the Classroom. (Newmann, Secada, 
& Wehlage, 1995). 

An aggregate score across three of the four scales was used as an overall measure of the Intellectual 
Challenge of instruction for each teacher. We omitted the rating for Substantive Conversation because 
ratings were highly influenced by the organizational structure of lessons. Specifically, lessons 
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involving teacher guided whole-group or small-group discussions yielded better opportunities to 
gather evidence on conversation than lessons with students working individually. Additionally, to 
enhance observer agreement for Intellectual Challenge ratings, we conducted training sessions for 
researchers immediately before each series of site visits began. We also utilized Many-Facets Rasch 
Analysis (Linacre, 2004) to adjust the measure of Intellectual Challenge for the relative severity (or 
leniency) of each observer during analyses. 

Table 4.5 reports the adjusted composite Intellectual Challenge scores for immersion and control 
teachers in fall 2004 and spring of 2005 and 2006. In fall, the sixth-grade control teachers' mean score 
(1.86) was significantly higher than the immersion teachers’ mean score (1.55). The difference 
favoring the control teachers represented a moderate effect size (-0.41). Thus, the sample of control 
teachers in fall engaged students in lessons involving a higher level of intellectual challenge. That is, 
lessons required a higher level of thinking, delved into topics more thoroughly, and made stronger 
connections with students’ background experiences and the world beyond the classroom. On the 
contrary, in spring 2005, sixth-grade teachers at immersion schools had a slightly higher mean score 
(1.84) than control teachers (1.78), but the difference between the groups was statistically 
insignificant. 


Table 4.5. Adjusted Intellectual Challenge Scores for Immersion and Control Teachers 


Group 

Immersion 

Control 

f-value 

P 

Effect 

Size 

N 

Mean 

SD 

N 

Mean 

SD 

Fall 2004“ 

69 

1.55 

0.67 

73 

1.86 

0.89 

-2.45 

0.020* 

-0.41 

Spring 2005“ 

110 

1.84 

0.92 

119 

1.78 

0.87 

0.47 

0.640 

0.06 

Spring 2006“ 

63 

1.72 

0.61 

58 

1.75 

0.72 

-0.24 

0.811 

-0.04 

Spring 2006 b 

67 

1.88 

0.88 

58 

1.93 

0.85 

-0.31 

0.756 

-0.06 


Note. Intellectual Challenge scores could range from 1 (lowest rating) to 5 (highest rating). *Difference is 
statistically significant. Effect size is Cohen’s d. The composite score does not include Substantive Conversation. 

“Sixth grade core-content teachers (reading/English language arts, math, science, and social studies). 
b Seventh grade core-content teachers. 

Findings for spring 2006 revealed very small differences between the mean Intellectual Challenge 
scores for sixth-grade teachers’ lessons at immersion (1.72) and control schools (1.75) or for seventh- 
grade teachers’ lessons (immersion, 1.88; control, 1.93). Across all classroom observations, lessons 
generally failed to intellectually challenge students, with ratings below 2 on the 5-point Intellectual 
Challenge scale. Seventh-grade teachers’ lessons were slightly more challenging than lessons in sixth- 
grade classrooms. In general, there was scant evidence that the availability of laptop computers and 
digital resources allowed students to experience more intellectually demanding work. 

In addition to analyses for all observed teachers, we also analyzed scores for teachers who were 
observed in fall 2004 and again in spring 2006. Results in Table 4.6 show that the two-year change in 
adjusted Intellectual Challenge scores for lessons was statistically insignificant for both teacher 
groups. Teachers at immersion schools showed a slight increase in the mean intellectual demand of 
lessons between fall 2004 (1.76) and spring 2006 (1.78), while the mean intellectual challenge of 
control teachers’ lessons decreased (from 1.99 to 1.69). Across all observations, the Intellectual 
Challenge ratings, which were below 2.0, indicated that the intellectual quality of lessons, on average, 
was extremely low. 
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Table 4.6. Adjusted Intellectual Challenge Scores for Sixth-Grade Immersion 
and Control Teachers with Pre- and Post-Measures 



Fall 2004 

Spring 2006 



Group 

N 

Mean 

SD 

Mean 

SD 

f-value 

P 

Immersion 

26 

1.76 

0.80 

1.78 

0.59 

0.10 

0.923 

Control 

25 

1.99 

0.84 

1.69 

0.70 

-1.53 

0.138 


Note. Intellectual Challenge scores could range from 1 (lowest rating) to 5 (highest rating). 
The composite score does not include Substantive Conversation. 


Conclusions 

Even though the level of implementation was generally low at many campuses in the second year, 
technology immersion positively affected teachers in a number of ways. Key findings are the 
following: 

• Immersion teachers grew in technology proficiency and in their use of technology for 
professional productivity at significantly faster rates than control teachers. 

• Immersion teachers expressed increasingly stronger ideological affiliations across time with 
classroom technology integration and learner-centered practices than control teachers. At the 
same time, immersion teachers reported generally low and stable resistance to technology. 

• Teachers in immersion schools collaborated more often with their peers on technology-related 
instructional and learning issues than control teachers, and students in immersion classrooms 
used technology applications more often for core-subject learning activities. 

• Across both treatment and control campuses, school poverty was negatively associated with 
teacher growth on several technology-related indicators. Most importantly, teachers in schools 
with above average levels of school poverty grew in technology proficiency at a significantly 
slower rate. 

• The availability of technology resources had little, if any, effect on the intellectual challenge 
of immersion teachers’ lessons. Across both immersion and control teachers’ classrooms, the 
intellectual demand of core-subject lessons was typically low. 
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5. Effects of Technology Immersion on Students and Learning 


In the theoretical model for technology immersion, we assume that an improved school environment 
for technology will lead to teachers who have greater technology proficiency and use technology more 
often for their own professional productivity. Moreover, in a fully immersed school, teachers have 
students use technology almost daily in their classrooms, and technology provides a means to enhance 
the intellectual challenge and relevance of lessons. Given changes in teacher knowledge and practices 
in an immersion school, we also reason that improved school and classroom environments for 
technology will lead students to greater technology proficiency and use, more frequent peer 
collaboration, opportunities for more challenging and relevant school work, stronger engagement in 
school and learning, and enhanced personal self-direction. 

Consistent with our suppositions, findings reported in Chapter 4 confirm that teachers at immersion 
schools have grown individually in important areas. Immersion teachers, in comparison to their 
control counterparts, arc more technically proficient, use technology more often for professional 
productivity, express a stronger ideological affiliation with immersion practices, and have their 
students use technology more often. On the contrary, results also show that student technology 
activities occur infrequently in immersion teachers’ core-content classrooms, and more importantly, 
new resources generally have failed to advance the intellectual quality of teachers’ lessons. 

Considering both the encouraging and less promising findings on second-year implementation of 
technology immersion, we investigate in this section the effects of immersion on students and their 
learning experiences. 


Immersion Effects on Student Mediating Variables 

Data on student mediating variables come from paper- and-pencil surveys {Student Questionnaire and 
Style of Learning Inventory) completed by two cohorts of students. Cohort 1 students completed 
surveys as sixth graders in fall 2004 and spring 2005, and again as seventh graders in spring 2006. 
Cohort 2 students completed surveys as sixth graders in fall 2005 and in spring 2006. The Student 
Questionnaire measures students’ technology proficiency, technology use, and views on technical 
problems. The questionnaire also gauges students’ opportunities to work with peers in small groups 
and their satisfaction with school. The Style of Learning Inventory ( SLI) measures various aspects of 
students’ self-directed learning. Overall, response rates for the Student Questionnaire were in the 80% 
to 90% range across time periods, with only slight differences in response rates between cohorts and 
comparison groups. Response rates for the SLI ranged from 77% to 82% across administrations. With 
the exception of spring 2005, there were only slight differences in response rates between cohorts and 
comparison groups. (See a complete list of response rates in the methodology chapter.) 

Sections to follow present findings for the two student cohorts. Results are reported for Cohort 1 
students who completed surveys at three time points. For this cohort, we use hierarchical linear 
modeling (HLM) growth models to examine the effects of technology immersion on students’ 
individual growth rates for various measures. Results also are presented for Cohort 2 students who 
completed surveys at two time points. For these students, we use HLM to estimate the effects of 
technology immersion on students’ spring 2006 scale scores. For both student groups, immersion 
effects are estimated for the following scales: Technology Proficiency, Self-Directed Learning, School 
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Satisfaction, Classroom Activities, Small-Group Work, and Technical Problems. Cronbach’s alpha 
coefficients (measures of internal consistency reliability) for student-level scales ranged from 0.80 to 
0.94. 


Cohort 1 Students (Seventh Graders) 

HLM Growth Analysis 

Longitudinal data allowed researchers to examine the nature of student change over time: fall 2004 
(baseline), spring 2005 (after the first implementation year), and spring 2006 (after the second 
implementation year). Analyses contrast the growth trajectories for Cohort 1 students at immersion 
and control schools. We analyzed immersion effects on students’ self-perceptions and technology- 
related activities using three-level hierarchical linear growth models. These HLM models produce 
student- and school-specific effects (i.e., the extent to which scale scores vary across time, students, 
and schools). 

Level 1: Repeated-measures model. Level 1 is a repeated-measures model (i.e., survey time within 
students) that enables us to capture key features of growth (e.g., initial status, rate of change). In the 
model, Y,u is the survey scale score at year t for student i in school j, and Survey Time is the point at 
which students completed surveys (0 = fall 2004, 1 = spring 2005, 2 = spring 2006). The key 
parameters in the model are 7r W; and jtuj- The coefficient ko\j represents the “initial status” (that is, the 
estimated initial scale score), for student i in school j in fall 2004, and 7r /i; is the growth rate (rate of 
change) for student i in school j per school year. The e tij is the error term (within- student measurement 
error) assumed to be normally distributed with a mean of 0 and a constant variance. Thus, at level 1, 
the model is 

Y,ij = n 0ij + 7i i ij (Survey Time) tij + e tij . . 

Level 2: Student-level model. The Level 2 model (between-students model) allows us to determine 
differences between students in features of growth (e.g., initial status \noij\, rate of change [ zr / f/ ] ) . In the 
student-level model, fiooj represents the mean initial status of a more advantaged student (advantaged = 
0, disadvantaged = 1 ) within school j, and f} IOj represents the mean rate of change for an advantaged 
student within school j. The coefficients fioij and fi / /; represent the effects of student poverty on initial 
status and school year rate of change, respectively. The r 0 ,j and r W; are residuals (i.e., random effects). 
At level 2, the model is 


n 0 y = Pooj + Poiji Disadvantaged) n + r 0ij 
nuj = pioj + pnj(Disadvantaged)jj + r n j. 


Level 3: School-level model. At the school level (level 3), we examine how students’ initial status 
(ftooj) and growth (fhop vary across schools as a function of school-level random effects (ju 0 oj and fijoj), 
as well as school conditions including immersion status (an indicator valuable with a value of 0 for a 
control school and a value of 1 for an immersion school) and school poverty (a continuous variable 
with percentages ranging from 33.7% to 100%, and with a grand mean of 68.6%). That is, we theorize 
that being in an immersion school is positively related to students’ growth on technology-related 
scores, after controlling for the poverty level of the school. Thus, we pose the following school-level 
model: 

Pooj = yooo + Yooi( Immersion status )j + y 0 02( School Poverty )j + nooj 

Pioj = yioo + yioi( Immersion status )j + y I02 (School Poverty )j + /Jioj. 
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In the model, y 0 oo is the overall mean initial status of an advantaged student at a control campus with 
an average level of school poverty, and y /00 is the overall mean student growth rate (of an advantaged 
student at a control campus with an average level of school poverty). The coefficients y 00 i and y m 
represent the direction and strength of association of immersion status on school-level initial status and 
growth rate, respectively. In addition, y 002 and y 102 represent the effect of school poverty on school- 
level initial status and growth rate, respectively. 

Immersion Effects 

Analyses involved the estimation of six, three-level HLM growth models. As shown in Table 5.1, we 
used separate models to estimate the effects of technology immersion on Cohort 1 students’ growth 
rates for their self-perceptions of Technology Proficiency, Self-Directed Learning, and School 
Satisfaction, as well as measures of their school technology experiences, including Classroom 
Activities, Small-Group Work, and Technical Problems. Analyses involved a total of 3,100 students 
who were continuously enrolled in schools since October 2004, with 1,454 at immersion schools and 
1,646 at control schools. 

Summary results show that technology immersion had positive student effects in a number of areas. 
After controls for school poverty (percentage of economically disadvantaged students) and student 
economic disadvantage (qualification for free- or reduced-price lunch). Cohort 1 students in 
immersion schools had significantly more positive growth trajectories than control students for three 
hypothesized effects of immersion. Estimated mean yearly rates of change for both advantaged and 
disadvantaged students revealed positive growth Lends favoring immersion students for Technology 
Proficiency, Classroom Activities, and Small-Group Work. Moreover, although immersion students 
used computers and the Internet more often than control students, there was no significant difference 
between groups in their tendencies for reporting Technical Problems. 

In contrast, the technology immersion model assumes that having daily access to and personal 
responsibility for laptop computers will allow immersion students to become more Self-Directed 
Learners and will increase their School Satisfaction. Contrary to expectations, we found that as 
students in both the treatment and control groups advanced from sixth to seventh grade, they reported 
being less self-directed learners and expressed less satisfaction with school, and there were no 
statistically significant differences between the comparison groups. 


Table 5.1. Cohort 1 (Seventh Graders): Immersion Effects on Estimated Mean Growth Rates 
for Student Mediating Variables 


Scale Scores 

Immersion 
Effect Net of 
Student and 
School 
Poverty 

Immersion 
Yearly Growth Rate 

Control 

Yearly Growth Rate 

Advantaged 

Students 

Dis- 

advantaged 

Students 

Advantaged 

Students 

Dis- 

advantaged 

Students 

Student Self-Perceptions 




Technology Proficiency (5-pt) 

Yes*** 

0.41 

0.47 

0.21 

0.27 

Self-Directed Learning (7-pt) 

No 

-0.09 

-0.12 

-0.07 

-0.10 

School Satisfaction (5-pt) 

No 

-0.08 

-0.05 

-0.09 

-0.06 

School Technology 




Classroom Activities (5-pt) 

Yes*** 

0.35 

0.41 

0.03 

0.09 

Small-Group Work (5-pt) 

Yes*** 

0.07 

0.13 

-0.11 

-0.04 

Technical Problems (5-pt) 

No 

0.18 

0.24 

0.10 

0.16 


Source: Student surveys completed in fall 2004, spring 2005, and spring 2006. 
Note. ***p < .001. Items measured on either a 5-point or 7-point scale. 
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Student responses to specific scales, as presented below, help to explain the outcomes for both 
economically advantaged and disadvantaged students in immersion and control schools. 


Student Self-Perceptions 

Statistical details for the HLM growth models related to student self-perceptions of their technology 
proficiency, self-directed learning, and school satisfaction are reported in Table 5.2. Results show that 
across the student mediating variables, the extent of school poverty had either no discernable effect or 
a minimally significant association with students’ initial status and growth rates. In contrast, a 
student’s poverty (i.e., economically disadvantaged status), was a significantly negative predictor of 
students’ initial status for Technology Proficiency and School Satisfaction, but a significantly positive 
predictor of students’ rate of change for the two variables. Individual scales are discussed below. 


Table 5.2. Cohort 1 (Seventh Graders): Immersion (Fixed) Effect Analyses for Student Self- 
Perception Variables 



Technology Proficiency 

Self-Directed Leaming d 

School Satisfaction 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Initial status (fall 2004) 

3.031 

51.89*** 

4.551 

107.67*** 

3.807 

92.69*** 

Immersion 

-0.037 

-0.44 

0.105 

2.08* 

0.047 

1.18 

School Poverty 

0.000 

-0.14 

0.003 

2.26* 

0.000 

-0.21 

Student Disadvantage 

-0.361 

_9 JT*** 

-0.039 

-L07 

-0.091 

-2.89** 

Growth rate 

0.210 

8.08*** 

-0.068 

-3.59** 

-0.094 

-4.92*** 

Immersion 

0.198 

4.53*** 

-0.025 

-L04 

0.016 

0.72 

School Poverty 

-0.001 

-0.52 

-0.001 

-0.82 

0.001 

2.23* 

Student Disadvantage 

0.060 

3.49** 

-0.029 

-1.60 

0.033 

2.18* 


*p < .05; **p < .01; ***p < .001. 

d Immersion students had significantly higher initial self-directed learning scores. A latent variable regression, 
controlling for the effect of this initial difference on the growth rate, indicated that the difference between the 
original and adjusted immersion coefficients was not significant (the difference divided by the standard error of the 
difference = 1.44). 

Technology Proficiency. As a measure of their Technology Proficiency, students rated their skills in 
using technology applications on a 5-point scale ranging from 1 (/ can do this not at all or barely ) to 5 
(/ can do this extremely well). Students indicated their skill level on statements aligned with the Texas 
Technology Applications Standards. Results in Figure 5.1 compare the Technology Proficiency 
growth trajectories for students in immersion and control schools. As the figure shows, both 
advantaged and disadvantaged immersion students, grew in technology proficiency at a significantly 
faster rate than their student counterparts in control schools. The yearly rates of change in proficiency 
for economically advantaged and disadvantaged students in immersion schools were 0.41 and 0.47 
scale-score points, respectively. This compares to 0.21 and 0.27 scale-score points, respectively, for 
advantaged and disadvantaged control students. Thus, economically disadvantaged students in 
immersion schools who began in fall 2004 with lower levels of technology proficiency surpassed even 
advantaged control students in proficiency by the end of seventh grade. 
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Figure 5.1 . Estimated mean Technology Proficiency growth trajectories for economically 

advantaged and disadvantaged student groups in immersion and control schools. 

Self-Directed Learning. Self-direction, as measured by the SLI for this study, includes statements 
relative to students’ forethought (e.g., goal setting, strategic planning, self-efficacy beliefs, intrinsic 
effort), performance/volition control (e.g., attention focusing, self-monitoring, and help seeking), and 
self-reflection (e.g., self-evaluation, adaptivity). Although prior research suggests that the 
individualized learning opportunities allowed through one-to-one technology will positively affect 
students’ self-regulated learning, our results revealed no significant immersion effect on students’ 
growth in self-direction. Initially, all sixth graders expressed ambivalent affiliations with self- 
regulating statements such as, “When a big project or report is assigned, I make a mental or written 
schedule to make sure everything gets done on time.” Students, on average, reported that such 
statements were, at best, only a little true. As both immersion and control students progressed through 
seventh grade, their responses to statements revealed a significantly negative growth trend. The 
estimated yearly rates of change in self-direction for advantaged and disadvantaged students in 
immersion schools were -0.09 and -0.12 scale-score points, respectively, compared to -0.07 and -0.10 
scale-score points, respectively, for their control-group counterparts. Overall findings suggest that 
seventh graders generally do not consider themselves to be strongly self-directed learners. 

School Satisfaction. Students also rated their level of School Satisfaction by indicating the extent of 
their agreement with statements on a 5-point scale ranging from 1 ( strongly disagree ) to 5 ( strongly 
agree). For example, students responded to items measuring their satisfaction with class work, the 
meaningfulness of class work, and the extent to which they perceived their class work to be useful to 
them in the future. As sixth graders, both immersion and control students generally agreed with 
statements measuring their school satisfaction. Flowever, both treatment- and control-group students 
reported significantly lower levels of school satisfaction across time. The estimated yearly rates of 
change in satisfaction for immersion and control schools were nearly identical (-0.08 and -0.09 scale- 
score points, respectively, for advantaged students; -0.05 and -0.06 for disadvantaged students). 
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School Technology 


Statistics for the HLM growth models that estimate effects on students’ technology experiences are 
reported in Table 5.3. Coefficients show that the level of school poverty had no significant association 
with either students’ initial status or their growth rates for technology -related scores. On the other 
hand, a student’s poverty was a significantly positive predictor of students’ growth in the frequency of 
classroom activities involving technology, as well as increased technical problems using computers. 
Findings for specific scales are discussed below. 


Table 5.3. Cohort 1 (Seventh Graders): Immersion (Fixed) Effect Analyses of School 
Technology Variables 


Dependent variable 
and predictor 

Classroom Activities 
(with technology) 

Small-Group Work 

Technical Problems 3 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Initial status (fall 2004) 

2.039 

33.02*** 

2.826 

51.64*** 

2.362 

39 . 94 *** 

Immersion 

0.133 

1.64 

0 . 00 1 _ 

0 . 0 1 _ 

-0.140 

- 1 . 88 * 

School Poverty 

- 0.002 

-0.72 

- 0.002 

-0.90 

-0.003 

-1.17 

Student Disadvantage 

0.005 

0.17 

-0.049 

- 0.86 

-0.078 

-1.33 

Growth rate 

0.03 1_ 

1.17 

-0.105 

-3.89** 

0.100 

3.30** 

Immersion 

0.323 

5.46*** 

0.173 

4.05*** 

0.084 

1.55 

School Poverty 

0.000 

0.09 

0.000 

- 0.20 

0.001 

0.36 

Student Disadvantage 

0.058 

2.83** 

0.063 

1 . 86 * 

0.060 

2.08* 


f p < .10; *p < .05; **p < .01; ***p < .001. 

■Immersion students had borderline significantly lower initial technical problems scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, indicated that the difference 
between the original and adjusted immersion coefficients was not significant (the difference divided by the 
standard error of the difference = 1.76). 

Classroom Activities. Students reported the frequency with which their teachers had them use 
specific technology applications (e.g., use a word processor for writing, use a spreadsheet to calculate 
or graph, create a presentation) in their English language arts, mathematics, social studies, and science 
classes combined. Students reported the frequency of technology use on a 5-point scale ranging from 
1 (never) to 5 ( almost daily). As anticipated given the increased availability of hardware and software 
in immersion schools, treatment students had a significantly steeper growth rate for their frequency of 
technology use in core-subject classes (see Figure 5.2). 
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Figure 5.2. Estimated mean growth trajectories for the frequency of Classroom Activities by 
economically advantaged and disadvantaged student groups in immersion and control schools. 

The yearly rates of change in Classroom Activities involving technology for economically advantaged 
and disadvantaged immersion students were 0.35 and 0.41 scale-score points, respectively. In 
contrast, advantaged and disadvantaged control students had relatively flat rates of change for 
classroom technology use (0.03 and 0.09 scale-score points, respectively). Despite significant 
increases in technology use by immersion students, however, mean use statistics suggest that students 
use various technology applications infrequently (about once or twice a month). 

Small-Group Work. Research studies consistently link one-to-one technology with a more 
collaborative classroom environment. Thus, our survey asked students to rate the frequency of their 
small-group interactions with classmates. Students rated statements, such as “we tutor or coach each 
other,” “brainstorm solutions to problems,” and “discuss assignments” on a 5-point scale ranging from 
1 {never) to 5 {almost daily). Growth rate coefficients show that students in immersion schools 
reported increasing opportunities for small-group work with peers across time (significant yearly 
growth rates of 0.07 and 0.13 scale-score points for advantaged and disadvantaged students, 
respectively). Quite the opposite, students at control campuses reported less frequent small-group 
activities across survey times (negative yearly growth rates of -0.1 1 and -0.04 scale-score points for 
advantaged and disadvantaged students, respectively). 

Technical Problems. Given the increased availability of technology in immersion schools and 
classrooms, we reasoned that students might encounter more technical problems. Thus, we asked 
students to indicate about how often various Technical Problems happened when they tried to use a 
computer at school. Students rated the frequency of problems on a 5-point scale As a whole, students 
reported significantly more frequent problems with technology as they advanced from sixth to seventh 
grade, but there were no significant differences between the growth in technology-related problems for 
immersion and control students. 

Cohort 2 Students (Sixth Graders) 

We analyzed the effects of immersion on Cohort 2 students’ scale scores for mediating variables using 
two-level hierarchical linear models. 
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HLM Analyses 


Level 1: Student-level model. In the student-level model, spring 2006 scale scores from surveys were 
regressed on fall 2005 scale scores, economic status (0 if not disadvantaged, 1 if disadvantaged), 
African American status (0 if not African American, 1 if African American), Hispanic status (0 if not 
Hispanic, 1 if Hispanic) and gender (0 if male, 1 if female). That is, 

Yy = p 0 j + fii/Fall 2005 scale score) + fr/Disadvantaged) + fa/ African American ) + 

Pfr Hispanic) + [//Female) + r,y. 

Level 2: School-level model. A school-level model was developed to answer the question of whether 
immersion schools had higher scale scores than control schools, after controlling for initial scale 
scores, economic status, ethnicity, gender, and school-level poverty. That is, 

Poj = )’oo + yo/Immersion dummy) + y 02 (School Poverty) + /u 0 j. 

Immersion was an indicator variable with a value of 1 for an immersion school and a value of 0 for a 
control school. School poverty was a continuous variable with percentages ranging from 33.7% to 
100%, and a grand mean of 68.6%. 

Immersion Effects 

Analyses involved the estimation of six, two-level HLM models. As shown in Table 5.4, we used 
separate models to estimate the effects of technology immersion on Cohort 2 students’ spring 2006 
scores for their self-perceptions of Technology Proficiency, Self-Directed Learning, and School 
Satisfaction, as well as measures of school technology, including Classroom Activities, Small-Group 
Work, and Technical Problems. Analyses involved a total of 4,033 students who were continuously 
enrolled in schools since October 2005, with 1,988 at immersion schools and 2,045 at control schools. 


Table 5.4. Cohort 2 (Sixth Graders): Immersion Effects on Student Mediating Variables 



Immersion Effect Net of Fall 



Score, Student Demographic 
Characteristics, & School 

Magnitude of 
Effect (d) in Standard 

Scale 

Poverty 

Deviation Units 

Student Self-Perceptions 



Technology Proficiency 

Yes** 

0.30 (small) 

Self-Directed Learning 

No 

-0.03 (trivial) 

School Satisfaction 

No 

-0.01 (trivial) 

School Technology 



Classroom Activities 


0.83 (large) 

Small-Group Work 

Yes** 

0.25 (small) 

Technical Problems 

No 

0.05 (trivial) 


Source: Student surveys conducted in fall 2005 and spring 2006. 

Note. **p < .01; ***p < .001. Effect size is Cohen’s d value. The interpretation is that an effect size 
greater than 0.5 is large, 0.5-0. 3 is moderate, 0.3-0. 1 is small, and anything smaller than 0.1 is trivial. 


Summary results presented in Table 5.4 for Cohort 2 sixth graders show that technology immersion 
had significantly positive effects for the same mediating variables as reported for Cohort 1 students 
(seventh graders). After adjusting for fall 2005 scale scores, student demographic characteristics 
(gender, ethnicity, economic disadvantage), and school poverty (percentage of economically 
disadvantaged students), technology immersion had a significantly positive effect on students’ 2006 
scale scores for Technology Proficiency, Classroom Activities, and Small-Group Work. The 
immersion effect was larger for the frequency of Classroom Activities with technology (effect size 
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[ES] = 0.83) and smaller for students’ ratings of their Technology Proficiency (ES = 0.30) and 
frequency of Small-Group Work (ES = 0.25). Conversely, there were no significant effects of 
immersion on Cohort 2 students’ Self-Directed Learning. School Satisfaction, or frequency of 
Technical Problems. Statistical details for HLM analyses arc provided in Tables 5.5 and 5.6. 

In addition to differences between immersion and control groups, results for HLM analyses reported in 
Tables 5.5 and 5.6 revealed notable associations between the demographic characteristics of sixth 
graders and technology-related variables. Results indicate that: 

• Female students (net of fall scores, economic status, ethnicity, and school poverty) had 
significantly higher spring 2006 scores than males for Technology Proficiency and School 
Satisfaction, as well as the frequency of Small-Group Work in core classes. 

• Economically disadvantaged students (net of fall scores, gender, ethnicity, and school 
poverty) had significantly lower spring 2006 scores than advantaged students for Technology 
Proficiency and School Satisfaction. Additionally, being an economically disadvantaged 
student was negatively associated with almost all mediating variables. 

• Hispanic students (net of fall scores, gender, economic status, and school poverty) had 
significantly lower spring 2006 scores than non-minority students on Technology 
Proficiency, but significantly higher ratings of School Satisfaction as well as the frequency 
of Small-Group Work in core-subject classes. 

• African American students (net of fall scores, gender, economic status, and school poverty) 
had significantly lower spring 2006 scores than non-minority students on Technology 
Proficiency, but students reported a significantly greater frequency of Classroom Activities 
involving technology and Small-Group Work in core-subject classes. 

Findings for the second year continue to show that females rate their technology proficiency higher 
than males, and females express greater satisfaction with the kind of academic work they do in middle 
schools. Moreover, although immersion is closing the technology equity gap between advantaged and 
disadvantaged students, overall results show that economically disadvantaged students and more 
impoverished schools remain at risk due to lower student technical proficiency, lower levels of school 
satisfaction, and lower levels of technical support for computer use. For the ethnic groups, outcomes 
are mixed but suggest that students’ school and classroom experiences differ. 


Table 5.5. Cohort 2 (Sixth Graders): Immersion (Fixed) Effect Analyses of Student 
Self-Perception Variables 



Technology Proficiency 

Self-Directed Learning 

School Satisfaction 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

3.229 

60.01*** 

4.593 

155.85*** 

3.688 

110.37*** 

Immersion 

0.251 

3.52** 

-0.023 

-0.73 

-0.008 

-0.23 

School poverty 

0.001 

0.71 

0.001 

1.69 + 

-0.001 

-1.20 

Female 

0.074 

4 19*** 

0.030 

1.15 

0.113 

4 44*** 

Hispanic 

-0.064 

-2.31* 

-0.029 

-0.98 

0.068 

2.28* 

African American 

-0.111 

-2.29* 

0.037 

0.90 

0.073 

1.00 

Disadvantaged 

-0.125 

-3.63** 

-0.027 

-1.03 

-0.092 

-3.42** 

Fall 2005 score 

0.477 

20.72*** 

0.556 

27.70*** 

0.345 

15.23*** 


f p < .10; *p < .05; **p < .01; ***p < .001. 
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Table 5.6. Cohort 2 (Sixth Graders): Immersion (Fixed) Effect Analyses of School 
Technology Variables 



Classroom Activities 
(with technology) 

Small-Group Work 

Technical Problems 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Gamma 

Coefficient 

f-value 

Intercept 

2.118 

34 77 *** 

2.610 

45.61*** 

2.361 

39.92*** 

Immersion 

0.663 

7 i7*** 

0.214 

3.17** 

0.046 

0.61 

School poverty 

0.003 

1.03 

0.001 

0.26 

0.005 

2 . 22 * 

Female 

-0.003 

-0.13 

0.096 

2.91** 

0.013 

0.49 

Hispanic 

0.048 

1.33 

0.092 

2.06* 

-0.055 

-1.49 

African American 

0.142 

3.15** 

0.197 

3.75*** 

0.074 

1.17 

Disadvantaged 

0.014 

0.42 

-0.009 

-0.24 

-0.043 

-1.38 

Fall 2005 score 

0.234 

16.55*** 

0.227 

11.65*** 

0.202 

7 42 *** 


f p < .10; *p < .05; **p < .01; ***p < .001. 


Immersion Effects on Student Engagement 

Based on our literature review, we theorized that students who attended schools immersed in 
technology would express higher levels of school satisfaction. Additionally, we anticipated that 
increased technology access and use would lead to improved student conduct, and consequently, fewer 
discipline problems and increased school attendance. Findings on student engagement presented below 
show that students in immersion schools exhibited significantly stronger school engagement on one 
indicator and significantly less engagement on another. 

School Satisfaction 

Students’ satisfaction with school provides one measure of engagement, and as reported earlier in the 
FILM analyses, there was no significant immersion effect on students’ School Satisfaction for either 
Cohort 1, seventh graders or Cohort 2, sixth graders. Students in immersion and control schools 
expressed correspondingly modest levels of satisfaction with the kinds of work they do in classes and 
the relevance of their schoolwork (about 3.7 on the 5-point scale, on average). 

Student Discipline and Behavior 

As another measure of engagement, we collected student-level data from schools on disciplinary 
actions occurring during the 2005-06 school year. Texas requires that schools report each disciplinary 
action that results in a removal of a student from their regular academic program for a full school day. 
Accordingly, we compared the frequency of the Disciplinary Action Reports (PEIMS 425 records) 
submitted for each student during the 2005-06 school year for treatment and control schools. Findings 
for student cohorts presented in Figure 5.3 show that students in immersion schools had 
proportionately fewer behavioral and disciplinary problems than their counterparts in control schools. 
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□ Immersion □Control 


Figure 5.3. Number of students and average number of disciplinary actions for Cohort 1 (7th 
graders) and Cohort 2 (6th graders) attending immersion and control schools. 

Cohort 1 seventh graders at immersion schools had significantly fewer disciplinary actions than 
students at control schools ( t - 2.24, p < 0.03). Specifically, 2,929 control-group students had an 
average of 0.90 disciplinary actions compared to 2,569 immersion students who had an average of 
0.76 disciplinary actions. The effect size for the mean difference (0.14), however, was extremely small 
(, d - 0.06). Similarly, Cohort 2 sixth graders at immersion schools had significantly fewer disciplinary 
actions than students at control schools (t - 3.04, p < 0.002). In particular - , 2,830 control-group 
students had an average of 0.69 disciplinary actions, and 2,686 immersion students who had an 
average of 0.53 disciplinary actions. Again, the effect size for the mean difference (0.16) was very 
small (d - 0.08). 

Overall, second-year findings on student discipline and behavior mirror results for the first project 
year. Evidence shows that both sixth and seventh graders attending technology immersion schools had 
fewer disciplinary and behavior problems than their counterparts in control schools. Although the 
estimated size of the differences between comparison groups is considered statistically trivial, having 
fewer disciplinary actions per student in middle schools may have practical benefits in terms of day- 
to-day time and effort. 

Student Attendance 

School attendance rates (absolute values). Another indicator of school engagement is students’ 
school attendance. Accordingly, we compared the annual attendance rates for Cohort 1 immersion- 
and control-group students for the year prior to project implementation and for the two implementation 
years. We compared Cohort 2 students’ attendance for the year prior to project implementation and 
after one implementation year. Results in Table 5.7 show that the average attendance rates of students 
in immersion schools are approximately 0.3 to 0.8 percentage points lower than the attendance rates of 
control students. The attendance-rate gap was present before project implementation and the difference 
widened after the implementation of technology immersion. 
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Table 5.7. School Attendance Rates for Cohort 1 and Cohort 2 Students 



Immersion 

Control 


Year/Grade 

Mean SD 

Mean 

SD 

Difference 

Cohort 1 (7th) 






2003-04 

97.09 

3.72 

97.36 

3.25 

-0.27 

2004-05 

96.57 

4.00 

96.82 

3.48 

-0.25 

2005-06 

95.67 

5.21 

96.47 

4.07 

-0.80 

Cohort 2 (6th) 






2004-05 

96.89 

4.04 

97.19 

3.37 

-0.30 

2005-06 

96.19 

4.54 

96.75 

3.82 

-0.56 


Source: Individual student data from TEA. 

Note. Cohort 1 included 1,978 immersion and 2,320 control students with attendance data 
from 2004 to 2006. Cohort 2 included 2,287 immersion students and 2,485 control students 
with attendance data for 2005 and 2006. 

HLM analyses of attendance. To test the effects of immersion on student attendance, while 
controlling for school and student characteristics, we conducted HLM analyses. For Cohort 1, 
longitudinal data allowed researchers to use a three-level HLM growth model to examine changes in 
school attendance rates over time: 2003-04 (baseline), 2004-05 (after one year), and 2005-06 (after 
two years). For Cohort 2 students, we used a two-level HLM model to examine the effects of 
immersion on schools’ 2005-06 attendance rate. Table 5.8 presents the HLM statistics for both student 
cohorts. 


Table 5.8. Immersion (Fixed) Effect Analyses of Student Attendance 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

t 

Cohort 1 (7th Graders) 

3-Level HLM Growth Model 

Initial attendance (2004) 

97.076 

616.69*** 


Immersion 

-0.053 

-0.24 


School poverty 

0.022 

4 41 *** 


Disadvantaged 

-0.702 

-5.31*** 


Growth rate 

-0.355 

-5 27*** 


Immersion 

-0.217 

-2.18* 


School poverty 

-0.005 

-1.84' 


Disadvantaged 

-0.154 

-1.86’ 

Cohort 2 (6th Graders) 




2-Level HLM Model 

Intercept 

96.922 

520.22*** 


Immersion 3 

-0.268 

-1.83 + 


School poverty 

0.001 

0.25 


Prior attendance 

0.627 

20.73*** 


Disadvantaged 

-0.560 

-3.83*** 


Female 

0.242 

1.81 : 


Hispanic 

0.115 

0.55 

African American -0.329 -0.85 


f p < .10; *p < .05; **p < .01; ***p < .001. 

J The effect size was 0.07. Although statistically significant, this effect was trivial in magnitude. 

The interpretation is that an effect size greater than 0.5 is large, 0.5-0. 3 is moderate, 0.3-0. 1 is 
small, and anything smaller than 0. 1 is trivial. 

Contrary to expectations, results indicate that technology immersion has a significantly negative effect 
on students’ school attendance. Analyses for Cohort 1 (seventh graders) estimated immersion effects 
on schools’ adjusted average rates of attendance (controlling for student and school poverty). As 
Figure 5.4 illustrates, average school attendance rates for economically advantaged immersion- and 
control-group students in schools with average rates of school poverty decreased as students advanced 
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from fifth to seventh grade. The yearly estimated negative rate of change in attendance for immersion 
students (-0.57 percentage points) was greater than the annual change for control students (-0.36 
percentage points). Thus, at the end of seventh grade, advantaged students in immersion schools had 
an estimated average attendance rate of 95.88% percent compared to 96.37% for control students, with 
the statistically significant difference favoring control students (t - -2. 1 8, p < .05). Attendance rates 
for economically disadvantaged students decreased at an even faster pace, with yearly negative change 
rates for disadvantaged students in immersion schools greater than the declining rates for control 
students (-0.73 percentage points versus -0.51, respectively). 


Attendance 

Rate 



♦ Immersion: Advantaged Students 5K Immersion: Disadvantaged Students 

— -A — Control: Advantaged Students — -X — Control: Disadvantaged Students 


Figure 5.4. Estimated attendance rates for economically advantaged and disadvantaged Cohort 1 
students in immersion and control schools with average rates of school poverty. 


Results for Cohort 2 students presented in Table 5.8 show that average school attendance rates for the 
sixth-grade students declined as they moved from elementary to middle school. The difference in 
estimated average attendance rates for students in immersion and control schools is statistically 
significant (t = -1.83, p < .10), but the size of the difference (-0.27 percentage points) is practically 
very small. 


Overall, there was no “boost” in school attendance for students in immersion schools. Instead, school 
attendance decreased at immersion campuses and the attendance gap between students in immersion 
and control schools increased across school years. It is difficult to explain why the presence of one-to- 
one technology would have a negative effect on student attendance. Further investigation of student 
attendance patterns will be needed to provide insight into student behaviors. 

Conclusions 


In the second project year, we investigated the effects of technology immersion on students and 
learning for Cohort 1 (seventh graders) and Cohort 2 (sixth graders). Data from two student cohorts 
allowed researchers to examine the constancy of results across groups, and indeed, we found 
consistent outcomes for mediating variables. Key findings are the following. 
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Technology immersion positively affects students’ technology proficiency. Immersion 
students grew in proficiency at a significantly faster rate or to a higher level than control 
students. Immersion substantially closed the technology equity gap for economically 
disadvantaged students. 

Technology immersion positively affects students’ classroom technology use and interactions 
with peers. Students in immersion schools used various technology applications significantly 
more often in their core-subject classrooms than control students, and they had significantly 
more frequent opportunities to learn in small groups with their classmates. 

Even though immersion students had increased access to technology and used technology 
more often, they reported technical problems when using computers at school at a similar 
frequency as control students. 

Immersion students, who had access to personal laptop computers and resources for learning, 
regarded themselves as no more self-directed learners than control students. As both 
immersion and control students progressed from sixth to seventh grade they reported 
significantly less self-directed learning behaviors. 

Technology immersion positively affects student discipline and behavior. Students in 
immersion schools, on average, had proportionately fewer behavioral and disciplinary 
problems that removed them from the regular academic program than their counterparts in 
control schools. 

The school attendance rates for immersion students were significantly lower than attendance 
rates for control-group students. 

Immersion and control-group students expressed similar levels of school satisfaction, with 
both groups reporting significantly lower levels of school satisfaction across time. 



6. Effects of Technology Immersion on Student Achievement 


Technology immersion aims ultimately to increase middle school students’ achievement in core 
academic subjects (English language arts, mathematics, science, and social studies) as measured by the 
Texas Assessment of Knowledge and Skills (TAKS). Accordingly, we theorize that students who are 
enrolled in fully immersed schools will experience school and classroom environments that promote 
stronger individual learning and technology proficiencies, more intellectually challenging work, and 
enhanced engagement in school and learning. In turn, changes in students and their learning 
experiences will contribute to enhanced performance on state assessments. In the second year of the 
technology immersion project, as detailed in previous report chapters, we have cited noteworthy 
outcomes in some areas (e.g., increases in teachers’ technology knowledge and the frequency of 
classroom technology use, as well as improvements in students’ technology proficiency and school 
behavior). At the same time, we have noted that low project implementation levels at many schools 
may have diminished immersion’s potential impact on student achievement during the project’s 
second year. Given existing contextual conditions, the following sections present academic 
achievement results for Cohort 1 and Cohort 2 students who were enrolled continuously in the 2 1 
immersion and 21 control schools through TAKS testing in April 2006. 

Texas Assessment of Knowledge and Skills 
Passing Standards and Scale Scores 

The TAKS is Texas’ criterion-referenced assessment that measures students’ mastery of the state’s 
content standards, the Texas Essential Knowledge and Skills (TEKS). The TAKS assesses reading and 
mathematics at grades 6 and 7 and writing at grade 7. This study uses several types of TAKS scores. 

• Met the standard. This score represents satisfactory academic achievement. Students who 
meet this standard performed at a level that was at or somewhat above the state passing 
standard. Thus, students demonstrated a sufficient understanding of the knowledge and skills 
measured at the grade level. 

• Commended performance. This score represents high academic achievement. Students who 
meet this standard performed at a level that was considerably above the state passing standard. 
Therefore, students demonstrated a thorough understanding of the knowledge and skills 
measured at the grade level. 

• TAKS scale score. The scale score is a statistic that provides a comparison of scores with a 
standard set at 2100 for each grade level. The scale score can be used to determine whether a 
student met the minimum standard or achieved commended performance, but it cannot be 
used to evaluate a student’s progress across grades or subject areas. TAKS scale scores are 
used to calculate standardized scores for this study. 

Texas has adopted a phase-in plan for implementing increasingly rigorous passing standards on the 
TAKS. In 2002-03, passing was set at two standard errors of measurement (SEM) below the passing 
standard recommended by the State Board of Education panel. In 2004-05, passing standards for 
reading, mathematics, and writing were fully implemented. Thus, for this study, TAKS scores for 
2003-04 have been converted to reflect the panel recommended passing standard. 
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Standard Scores 


In addition to the scores provided by the TEA, researchers generated standard scores that are used to 
compare student progress on TAKS across grade levels. A standardized score — or z score — was 
calculated for each student and for every testing occasion and subject. The z score is calculated by 
subtracting the statewide mean grade-level scale score from each student’s scale score and dividing by 
the statewide scale score standard deviation. The z score, which has a mean of zero and a standard 
deviation of 1.0, indicates how many standard deviations from the mean a score lies. 

A major disadvantage of z scores is that about half of the scores are negative, and negative scores may 
be difficult to fully understand. To overcome this limitation, we have transformed students’ z scores 
into normalized scores, or T scores. T scores are scores with a mean of 50 and a standard deviation of 
10. Thus, a student who scores at the state average will have a TAKS T score of 50. A student who has 
a score of 60 will be one standard deviation above the state average, and a student who has a score of 
40 will be one standard deviation below the state average. 


Progress in Meeting TAKS Standards 
Cohort 1 (Seventh Graders) 

One measure of student academic outcomes is their progress toward meeting TAKS passing and 
commended performance standards. Information in Table 6.1 compares the absolute performance of 
Cohort 1 students in immersion and control schools across three TAKS testing years. 


Table 6.1. Cohort 1 (Seventh Graders): 

TAKS Passing and Commended Performance Rates for Reading and Mathematics 


TAKS Test 

Group 

N 

2004 
Grade 5 
Percent 

2005 
Grade 6 
Percent 

2006 
Grade 7 
Percent 

2006-2004 

Difference 

Met Standard 

1 

Reading 

Immersion 

1,569 

66.4 

74.1 

70.5 

4.1 


Control 

1,842 

71.8 

80.9 

75.7 

3.9 

Mathematics 

Immersion 

1,591 

68.9 

60.3 

63.0 

-5.9 


Control 

1,850 

71.4 

66.5 

65.1 

-6.3 

Commended 

Performance 

Reading 

Immersion 

1,569 

18.9 

27.5 

16.3 

-2.6 


Control 

1,842 

21.9 

34.1 

17.5 

-4.4 

Mathematics 

Immersion 

1,591 

23.1 

18.3 

10.1 

-13.0 


Control 

1,850 

23.2 

20.3 

8.9 

-14.3 


Source: Analysis of individual student data from TEA. 

Note. The 2004 passing rates are based on 2005 standards. Students had TAKS scores in 2004, 2005, 
and 2006 and attended the same schools. 


Results show that Cohort 1 seventh graders at immersion schools had somewhat lower passing rates in 
spring 2006 for TAKS reading and mathematics than students at control campuses. However, TAKS- 
score comparisons between 2004 (5th grade baseline) and 2006 (7th grade) revealed similar gains or 
losses across immersion and control groups, respectively, for both TAKS reading (4. 1 percentage 
points versus 3.9 points) and mathematics (-5.9 percentage points versus -6.3 points). Comparison- 
group trends for Commended Performance generally reflect results for students who met the grade- 
level passing standard. The percentage of students achieving commended TAKS standards, at both 
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treatment and control schools, varied greatly across testing periods. Students had the greatest difficulty 
meeting commended standards for mathematics. 


Cohort 1 students at immersion schools also had lower TAKS passing rates for writing in 2006 than 
control students, and students’ TAKS-score gains between 2003 (4th grade baseline) and 2006 (7th 
grade) were larger for control-group students (1 1.4 percentage points versus 9.6 points for immersion 
schools). Control students also achieved Commended Performance in writing at a higher rate and had 
stronger 4th-to-7th-grade gains than immersion students (25.5 percentage points versus 19.6 points). 


Table 6.2. Cohort 1 (Seventh Graders): 

TAKS Passing and Commended Performance Rates for Writing 


TAKS Test 

Group 

N 

2003 
Grade 4 
Percent 

2006 
Grade 7 
Percent 

2006-2003 

Difference 

Met Standard 

Writing 

Immersion 

1,445 

79.3 

88.9 

9.6 


Control 

1,681 

81.9 

93.3 

11.4 

Commended Performance 

Writing 

Immersion 

1,445 

11.1 

30.7 

19.6 


Control 

1,681 

11.4 

36.9 

25.5 


Source: Analysis of individual student data from TEA. 

Note. The 2003 passing rates are based on 2005 standards. Students had TAKS scores in 
2003 and 2006 and attended the same schools. 


Cohort 2 (Sixth Graders) 

Table 6.3 presents Cohort 2 students’ passing rates for TAKS reading and mathematics. Sixth graders 
at immersion schools had somewhat lower passing rates in spring 2006 for TAKS reading and 
mathematics than students at control campuses. However, TAKS passing rate differences between 

2005 (5th grade baseline) and 2006 (6th grade) favored students at immersion schools in both reading 
(20.3 percentage points versus 16.9 points for control students) and mathematics (-1.7 percentage 
points versus -4.1 points for control students). Trends for Commended Performance showed higher 

2006 achievement rates for control students, but year-to-year differences between treatment and 
control groups were mixed. 

Table 6.3. Cohort 2 (Sixth Graders in 2005-06) 

TAKS Passing and Commended Performance Rates for Reading and Mathematics 


TAKS Test 

Group 

N 

2005 
Grade 5 
Percent 

2006 
Grade 6 
Percent 

2006-2005 

Difference 

Met Standard 


Reading 

Immersion 

1,789 

65.8 

86.1 

20.3 


Control 

2,000 

73.3 

90.2 

16.9 

Mathematics 

Immersion 

1,805 

71.6 

69.9 

-1.7 


Control 

2,033 

77.7 

73.6 

-4.1 

Commended 

Performance 

Reading 

Immersion 

1,789 

16.4 

28.0 

11.6 


Control 

2,000 

18.9 

33.1 

14.2 

Mathematics 

Immersion 

1,805 

21.2 

20.6 

-0.6 


Control 

2,033 

24.6 

21.5 

-3.1 


Note. Students had TAKS scores in 2005 and 2006 and attended the same schools. 
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Altogether, TAKS passing rates for reading, mathematics, and writing provide important evidence that 
helps to understand student progress toward meeting state standards — however, additional statistical 
analyses are necessary to assess the effects of immersion on student achievement. 


Effects of Immersion on Academic Achievement 

Researchers used hierarchical linear modeling (HLM) to estimate the effect of immersion on student 
academic achievement. HLM is a “value added” methodology. That is, after controlling for students’ 
initial achievement and characteristics and accounting for variance at the student and school levels, 
researchers can assess the “value added” by the treatment. 

The analyses to follow contrast the achievement of immersion and control students before and after 
two school years of project implementation (Cohort 1, seventh graders) and one school year of 
implementation (Cohort 2, sixth graders). Immersion effects for Cohort 1 are estimated for TAKS 
reading, mathematics, and writing T scores. Longitudinal scores for TAKS reading and mathematics 
allowed researchers to examine changes in students’ achievement over time: 2004 (5th grade 
baseline), 2005 (6th grade), and 2006 (7th grade). These analyses estimated immersion effects using 
three-level HLM growth models. For TAKS writing. Cohort 1 students’ scores were available for two 
time points: 2003 (4th grade baseline) and 2006 (7th grade). Accordingly, data analysis involved a 
two-level HLM model. Similar to writing, the effects of immersion on Cohort 2, sixth graders’ TAKS 
reading and mathematics T scores were analyzed using two-level HLM models. Cohort 2 students had 
TAKS scores for 2005 (5th grade baseline) and 2006 (6th grade). (See Appendix E for technical detail 
on the HLM models.) 

Cohort 1 (Seventh Graders) 

TAKS Reading and Mathematics 

The analyses that follow contrast the estimated achievement growth trajectories for Cohort 1 students 
in immersion and control schools. Three-level HLM growth models allowed researchers to examine 
the extent to which student achievement varied across time, students, and schools. Given the 
complexity of interpreting growth models, we constrained our final models to include school and 
student predictors that exhibited strong associations with achievement (i.e., school and student 
poverty). In the HLM growth model, level 1 is a repeated-measures model (i.e., TAKS assessment 
time within students) that enabled us to capture the key features of growth (e.g., initial status, rate of 
change). Time is the point at which students completed surveys (0 = fall 2004, 1 = spring 2005, 

2 = spring 2006). The level 2 model (between-students model) allowed us to determine differences 
between students in features of growth (e.g., initial status, rate of change), after adjusting for students’ 
economic status (1 if economically disadvantaged [i.e., eligible for the federal free- and reduced-price 
lunch program], 0 if not). At the school level (level 3), we examined how students’ initial status and 
growth varied across schools as a function of school-level random effects, as well as school 
conditions, including immersion (1 = immersion, 0 = control group) and school poverty (percentage of 
economically disadvantaged students attending a school). School poverty rates ranged from about 34% 
to 100%, with a mean of 68.6%. Thus, we hypothesized that being in an immersion school is 
positively related to students’ growth in achievement, after controlling for the poverty level of the 
school. 

Table 6.4 shows the results of the HLM growth analysis for Cohort 1 students. We generated two 
separate models to estimate the effects of immersion on students’ growth in reading achievement and 
mathematics achievement. We estimated school mean rates of change for immersion and control 
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students, as well as the separate effects of student economic disadvantage and the school poverty 
concentration on reading and mathematics learning. Analyses involved approximately 1,590 
immersion students and 1,860 control students. Comparison groups had nearly equivalent proportions 
of students included in longitudinal analyses (about 66.5% for immersion and 65.5% for control). 


Table 6.4. Data for Cohort 1 (Seventh Graders): Effects of Immersion (Fixed) 
on TAKS Reading and Mathematics Achievement Growth Rates 



TAKS Reading 
N = 3,419 

TAKS Mathematics 
N = 3,450 

Dependent variable 
and predictor 

Gamma 

Coefficient 

t- value 

Gamma 

Coefficient 

f- value 

Initial mean status 
(2004 TAKS T score) 

53.834 

84.75*** 

52.813 

80.17*** 

Immersion 3 

-1.350 

-2.08* 

-1.074 

-1.30 

School poverty 

-0.064 

_4 7i*** 

-0.047 

-2.61* 

Economic disadvantage 

-6.384 

-9.58*** 

-4.558 

-9.01*** 

Growth rate 

-0.547 

-2.87** 

-0.315 

-1.19 

Immersion 3 

0.257 

1.05 

0.220 

0.49 

School poverty 

0.007 

0.84 

0.008 

0.61 

Economic disadvantage 

0.461 

2.41* 

-0.285 

-1.91 ' 


: p < .10; *p < .05; **p < .01; ***p < .001. 

3 Immersion students had significantly lower initial TAKS reading scores. A latent variable 
regression, controlling for the effect of this initial difference on the growth rate, indicated that the 
difference between the original and adjusted immersion coefficients was not significant (the 
difference divided by the standard error of the difference = 0.26). 

As can be seen in the table, the initial mean TAKS reading status for the reference group (an 
economically advantaged student in a control school with an average level of school poverty) is 
estimated at 53.83 (the mean 2004 TAKS reading T score). The coefficient representing immersion 
(-1.350) shows that students in immersion schools had somewhat lower initial TAKS reading T scores 
(51.74) than control students. Considering that differences among schools in students’ initial 
achievement may be related to subsequent rates of change, we used statistical tests to establish that 
those differences did not affect estimations of student growth. Coefficients for initial status also 
showed that economically disadvantaged students and students attending schools with above average 
levels of poverty started behind their more advantaged counterparts in reading ability (-6.384 and 
-0.064 T-score points, respectively). 

After controlling for student and school levels of poverty, results in Table 6.4 show that there was no 
statistically significant effect of immersion on students’ growth rate for TAKS reading scores. 
Statistics show that advantaged students in control schools (with average poverty) began with a mean 
TAKS T score of 53.83 and their reading achievement decreased by about 0.55 T-score point per year 
(significant coefficient of -0.547). The positive coefficient for immersion (0.257) shows that reading 
scores for advantaged students in immersion schools (with average poverty) decreased at a slightly 
slower rate compared to control schools — 0.29 T-score point per year (-0.547 + 0.257 = -0.290). 
Economically disadvantaged students in both immersion and control schools grew in reading 
achievement at a significantly faster rate than their more advantaged peers. Figure 6.1 illustrates the 
estimated mean TAKS reading growth trajectories for advantaged and disadvantaged students by 
school comparison group. 
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Figure 6.1. Estimated mean TAKS reading achievement growth trajectories for economically 
advantaged and disadvantaged student groups in immersion and control schools. Differences 
between immersion and control groups are statistically insignificant. 

Results in Table 6.4 for Cohort 1 students’ initial status in TAKS mathematics generally mirror 
findings for reading. Control students in the reference group began with an estimated mean 
mathematics T score of 52.81. Their immersion student counterparts initially had a si mi lar estimated 
mathematics T score (51.74). Economically disadvantaged students and students attending schools 
with above average levels of poverty stalled significantly behind their more advantaged peers in math 
ability (-4.558 and - 0.047 T-score points, respectively). 

Similar to results for TAKS reading, after controlling for student and school levels of poverty, there 
was no significant effect of immersion on students’ growth rate for TAKS mathematics scores. 
Estimated math achievement for advantaged students in control schools (with average poverty) 
decreased by about 0.32 T-score point per year (coefficient of -0.315). The positive estimated 
coefficient for immersion (0.222) suggests that math scores for advantaged students in immersion 
schools (with average poverty) decreased at a slightly slower rate compared to control students — about 
0.10 T-score point per year. In contrast to reading, economically disadvantaged students at both 
immersion and control schools grew in mathematics achievement at a significantly slower rate than 
their more advantaged peers (about 0.29 T-score points less per year). Figure 6.2 shows the estimated 
mean TAKS mathematics growth trajectories for advantaged and disadvantaged students at immersion 
and control schools. 
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Figure 6.2. Estimated mean TAKS mathematics achievement growth trajectories for 
economically advantaged and disadvantaged student groups in immersion and control schools. 
Differences between immersion and control groups are statistically insignificant. 

TAKS Writing 

The effects of immersion on Cohort 1 students’ writing scores were analyzed using a two-level HLM 
model (see Table 6.5). In the student-level model (level 1), students’ 2006 writing T scores were 
regressed on 2003 writing scores (data from two years prior to the start of the technology immersion 
project), gender (1 if female, 0 if male), minority status (1 if Hispanic, 0 if not; 1 if African American, 
0 if not), and economic status (1 if economically disadvantaged, 0 if not). A school-level model (level 
2) answered the question of whether students in immersion schools had higher 2006 TAKS writing 
T scores than control-group students, after adjusting for initial achievement, student demographic 
characteristics, and school poverty. Immersion was an indicator variable with a value of 1 for an 
immersion school and a value of 0 for a control school. School poverty was a continuous variable 
indicating the percentage of economically disadvantaged students in a school (with a mean of 68.6%). 

Table 6.5. Data for Cohort 1 (Seventh Graders): 

Effects of Immersion (Fixed) on TAKS Writing Achievement 


TAKS Writing 
N= 3,126 

Dependent variable 
and predictor 

Gamma 

Coefficient 

f-value 

Intercept (TAKS T score) 

51.116 

78.70*** 

Immersion 

-0.907 

-1.47 

School poverty 

-0.011 

-0.63 

Female 

2.237 

6.29*** 

African American 

-2.241 

-4.05*** 

Hispanic 

-1.162 

-3.37** 

Economic disadvantage 

-1.712 

-3.38** 

Spring 2003 T score 

0.617 

30.28*** 


**p< .01; ***p < .001. 
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HLM analyses involved 1,445 immersion students and 1,681 control students (61.7% of students for 
immersion and 60.3% for control). Results for the analysis of the effects of immersion on Cohort 1 
students’ writing scores show that after controlling for students’ 2003 TAKS writing T scores, student 
demographic characteristics (gender, ethnicity, economic status), and campus poverty level, there were 
no significant differences in the 2006 writing T scores for students in immersion and control schools. 
Contrary to expectations, the coefficient shows that the immersion effect on writing is negative (about 
-0.91 T-score points lower than the control group), although the difference between groups is 
statistically insignificant {p — 0.149). 

Across both immersion and control schools, the demographic characteristics of students were strongly 
associated with TAKS writing achievement. Female students had significantly higher writing scores 
than males (about 2.24 T-score points); African American and Flispanic students had significantly 
lower writing scores than their non- minority group counterparts (-2.24 and -1.16 points, respectively); 
and economically disadvantaged students had significantly lower scores than their advantaged peers 
(-1.71 T-score points). Thus, students’ background characteristics mattered more than whether they 
attended an immersion or control school. 

Cohort 2 (Sixth Graders) 

TAKS Reading and Mathematics 

Immersion effects also were estimated for Cohort 2 students’ TAKS reading and mathematics T scores 
(see Table 6.6). We analyzed the effects of immersion on sixth graders’ reading and mathematics 
scores using separate two-level FILM models. In the student-level model (level 1), 2006 TAKS 
reading and mathematics T scores were regressed on 2005 reading and mathematics scores, gender (1 
if female, 0 if male), minority status (1 if Flispanic, 0 if not; 1 if African American, 0 if not), and 
economic status (1 if economically disadvantaged, 0 if not). A school- level model (level 2) estimated 
whether immersion schools had higher TAKS achievement scores than control schools, after 
controlling for initial achievement, student demographic characteristics, and school poverty. The 
immersion variable identified the comparison groups (1 if immersion, 0 if a control school). School 
poverty was a continuous variable depicting the concentration of economically disadvantaged students 
in a school. Analyses involved approximately 1,800 immersion students and 2,000 control students, 
with similar proportions of comparison-group students included in analyses (about 67.8% for 
immersion and 70.9% for control). (See Appendix E for technical details). 


Table 6.6. Data for Cohort 2 (Sixth Graders): 

Effect of Immersion (Fixed) on TAKS Reading and Mathematics Achievement 



TAKS Reading 

TAKS Mathematics 


A =3,789 

A = 3,838 

Dependent variable 

Gamma 


Gamma 


and predictor 

Coefficient 

t-value 

Coefficient 

t-value 

Intercept (TAKS T score) 

48.828 

108.41*** 

48.067 

97.03*** 

Immersion 

0.236 

0.55 

0.950 

1.27 

School poverty 

-0.025 

- 1 .91 1 

-0.020 

-1.11 

Female 

1.518 

7 1 j*** 

0.816 

3.56** 

African American 

-0.957 

-2.65** 

-1.961 

-3.80*** 

Hispanic 

-0.869 

-2.24* 

-0.752 

-2.52* 

Economic disadvantage 

-1.218 

-3.43** 

-1.027 

-4 00*** 

2005 TAKS T score 

0.648 

24.03*** 

0.680 

31.69*** 

Pretest x Immersion 

0.011 

0.34 

0.050 

1.97* 


'p < .10; *p < .05; **p < .01; ***p < .001. 
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TAKS reading outcomes for sixth graders reported in Table 6.6 show that after controlling for 
students’ prior reading achievement, demographic characteristics, and the level of school poverty, 
there were no significant differences in the 2006 TAKS reading T scores for students in immersion and 
control schools. The immersion effect on reading was positive (about 0.24 T- score points) but not 
statistically significant {p - 0.584). There was a slightly negative effect on TAKS reading achievement 
for sixth graders who attended schools with above average levels of school poverty (-0.03 T-score 
points). In contrast to school-level effects, students’ individual characteristics were strong predictors 
of reading achievement. Female students had significantly higher TAKS reading T scores, whereas 
minority and economically disadvantaged students had significantly lower TAKS reading scores. 

Results for sixth graders’ TAKS mathematics scores showed that after controlling for students’ prior 
math achievement, demographic characteristics, and school poverty, there was a significant immersion 
effect (0.050, p = 0.055) on students’ 2006 TAKS mathematics T scores which acted through the 2005 
TAKS math score. Figure 6.3 illustrates this effect for average immersion and control students. Other 
factors being equal, higher pretest math scores (2005 TAKS), predicted larger gaps in the posttest 
scores (2006 TAKS math) favoring immersion students. Thus, for mathematics, immersion had a 
stronger and significant effect for higher achieving students. 



Figure 6.3. Significant immersion effect on TAKS mathematics achievement, which acts through 
the pretest score. 

Conclusions 

In the second project year, we examined the effects of immersion on Cohort 1 students (seventh 
graders who were continuously enrolled in schools for two years) and Cohort 2 students (sixth graders 
who were continuously enrolled for one school year). Key findings are the following. 

• Results for Cohort 1 , seventh graders showed no statistically significant effects of immersion 
on TAKS reading, mathematics, or writing achievement. Although no significant effects were 
evident, positive estimated mean growth trajectories for immersion students in TAKS reading 
and mathematics suggested that the achievement gap between immersion and control schools 
narrowed slightly over time. Economically disadvantaged students in immersion schools 
appeared to benefit the most through significantly improved reading achievement. Outcomes 
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for TAKS writing favored students in control schools, although the difference was not 
statistically significant. 

Results for Cohort 2, sixth graders revealed no statistically significant effects of immersion on 
TAKS reading achievement. The immersion effect on reading was positive, but not large 
enough to differ significantly from control-group students. On the other hand, while there was 
no significant overall effect of immersion on TAKS mathematics achievement, immersion 
students who began with higher than average math pretest scores had significantly higher 
TAKS mathematics scores in 2006 than their control-group counterparts. 



7. Conclusions and Implications 


The evaluation of technology immersion (i.e., a laptop computer for every middle school student and 
teacher, wireless access throughout the campus, curricular and assessment resources, professional 
development and ongoing pedagogical support for curricular integration, and technical support for 
immersion) provides a scientific investigation of the effectiveness of technology immersion in 
increasing middle school students’ achievement in core academic subjects. This research is important 
because it provides a large-scale, methodologically rigorous study of the impacts of ubiquitous 
technology at a time when the number of one-to-one technology initiatives is steadily increasing 
(Penuel, 2006; Gewertz, 2007). Furthermore, the conceptual framework guiding the research allows a 
comprehensive examination of technology-infused school and classroom environments that builds on 
and extends the existing knowledge base. 

Given the limitations of the study’s quasi-experimental design, we analyzed extensive baseline data to 
establish the comparability of treatment and control groups. In the project’s second year, however, our 
research design was modified when two middle schools in one district (one immersion and one 
control) were excluded from analyses due to damage caused by Flurricane Rita on the Texas Gulf 
coast. Thus, second-year results are for the remaining 21 treatment and 21 control schools. A re- 
analysis of baseline data for the new sample revealed that school and student characteristics generally 
were unchanged and differences between groups remained statistically insignificant. Thus, the 
integrity of the study’s research design appears sound. Even though comparison groups are reasonably 
well matched, immersion schools have somewhat larger proportions of economically disadvantaged 
students, so we have used statistical methods to adjust for remaining differences that arise from 
sampling variability. On the whole, researchers are confident that reported effects can be attributed to 
the treatment. 

Our study, as designed, expanded to two student cohorts in the second year. Cohort 1 includes 5,538 
seventh graders (2,627 immersion, 2,911 control) who completed their second project; Cohort 2 
includes 5,507 sixth graders (2,685 immersion, 2,822 control) who finished their first year. Cohorts 
include predominantly minority (77%) and economically disadvantaged (74%) students. The middle 
schools are typically small (402 students, on average); however, enrollments vary widely (from 83 to 
1,447 students). Although schools are highly concentrated in rural and very small Texas districts, 
about a third of districts and schools are in large cities or suburban locations. 

Study Limitations 

Generalization of findings to a broader population is a primary study limitation. Compared to Texas 
middle-school students as a whole, students in the sample schools are substantially more Flispanic and 
less White and African American. Middle schools are also smaller than the statewide average (402 
students, on average, versus 667), and schools are located either in small or very small districts (64%) 
or large districts (36%). Additionally, for many variables, the study relies on self-reported data from 
surveys of teachers and students — thus, some findings on changes in proficiencies and practices reflect 
respondents’ perceptions. Nonetheless, the triangulation of evidence from multiple sources (surveys, 
classroom observations, state demographic and test databases, student cohorts) verifies the robustness 
of findings. 
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Criteria for Concluding Effects 


Until the 1990s, program effects were evaluated exclusively by testing the statistical significance of group 
differences. Since that time, scholars have recognized that statistical significance does not address (a) the 
importance of group differences, or (b) the replicability or truth of group differences (e.g„ Cohen, 1994; 
Schmidt, 1996). Increasingly, researchers have attended as well to the effect sizes that quantify the group 
differences (cf. American Educational Research Association, 2006; Grissom & Kim, 2005; Thompson, 
2006). Effect sizes quantify the degree to which group statistics, such as means, are unequal. Small effect 
sizes are noteworthy under certain conditions. 

Small effect sizes are noteworthy when evidence suggests that effects are replicable. This occurs, for 
example, when effects are replicated (a) across subject matters (e.g., both reading and mathematics), or 
(b) across time (e.g., both Cohorts 1 and 2). The statistical significance tests reported in Chapter 6, and 
discussed here, do not test either of these forms of effect replicability, and so cannot address the result 
replicability question. Small effects are also more noteworthy when (a) the outcome variable is 
particularly resistant to intervention (Prentice & Miller, 1992), or (b) small effects generated over time 
cumulate into large effects (Abelson, 1985), or both. An example of effect cumulation in the context of 
the current project would be if learning or student behavior improved not only in the project intervention 
years, but continued subsequently in future grade levels. Of course, only continuing evaluation can 
document cumulative effects over time. 

Summary of First- and Second-Year Findings 

Our first-year report — Evaluation of the Texas Technology Immersion Pilot: First-Year Results 
(Shapley et al., 2006a) — revealed positive effects of technology immersion on schools (leadership, 
innovative culture, and parent and community support), teachers (technology proficiency and 
productivity, technology use and integration, and peer collaboration), and students (technology 
proficiency and use, small-group work, school satisfaction, and behavior). In most cases, the sizes of 
effects were of both statistical and practical importance. Findings for the second implementation year 
relative to these same variables arc generally consistent with first- year results. Steadfast outcomes 
across two evaluation years and two student cohorts show that immersing a middle school in 
technology produces schools with stronger principal leadership for technology, greater teacher 
collaboration and collective support for technology innovation, and stronger parent and community 
support for technology. Additionally, teachers in immersion schools are more technically proficient 
and use technology more often for their own professional productivity, their students use technology 
more often in core-subject classrooms, and teachers adopt more integration-oriented and learner- 
centered ideologies. Students in immersion schools are more technically proficient, use technology 
more often for learning, interact more often with their peers in small-group activities, and have fewer 
disciplinary problems than control-group students. 

Also consistent with first-year results, we found no significant effect of technology immersion in the 
second year on student self-directed learning, and we found a significantly negative immersion effect 
on school attendance. Moreover, the availability of technology across two years provided no 
significant increase in the intellectual challenge of immersion teachers’ core-subject lessons. 

First-year findings on academic achievement revealed no statistically significant immersion effects on 
Texas Assessment of Knowledge and Skills (TAKS) reading or mathematics scores for Cohort 1, sixth 
graders. Similarly, second-year results for Cohort 1 students (as seventh graders) showed no 
significant effects of immersion on TAKS reading, mathematics, or writing achievement. Likewise, 
achievement results for Cohort 2 students (sixth graders involved in the project for one year) revealed 
no significant effect of immersion on TAKS reading achievement. However, for TAKS mathematics, 
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students in immersion schools who began the year with higher math pretest scores had significantly 
higher mathematics achievement than their control-group counterparts. The achievement gap favoring 
immersion students widened as pretest scores increased. Although TAKS score differences between 
immersion and control schools usually did not differ by statistically significant margins, second-year 
achievement trends, in contrast to first-year results, generally favored technology immersion schools. 

The study’ s overall outcomes as described above reflect the effects of immersion for schools that 
typically achieved low project implementation levels. We reported in the first year that 20 of the 22 
middle schools reached only partial immersion (2 on a 4-point implementation scale) rather than 
substantial (2 schools) or full immersion (no schools). Using somewhat different scoring criteria in the 
second year, we conclude that about a third of middle schools (6) achieved a substantial level of 
immersion, whereas the remaining two-thirds (15) reached minimal to partial immersion levels. 
Implementation evidence for this evaluation, consistent with other studies of whole-school reform, 
demonstrates that achieving quality implementation when the initiative involves fundamental school 
and classroom change is challenging (Borman, 2005; Vernez et al., 2006). Given existing contextual 
conditions, study limitations, and criteria for interpreting effects, major findings from the second year 
are described in the following section. 

Major Second-Year Findings 

Effects of Immersion on Teachers and Teaching 

In the second project year we assessed the effects of immersion on teachers and teaching by examining 
teachers’ rates of growth on mediating variables across three time points (fall 2004, spring 2005, and 
spring 2006). Analyses involved 802 teachers, including 364 in immersion schools and 438 in control 
schools. We observed classroom activities for a sample of core-subject teachers during the same time 
periods. 

Immersion teachers grew in technology proficiency and in their use of technology for 
professional productivity at significantly faster rates than control teachers. Technology 
immersion accelerated teachers’ growth in meeting the state’s Technology Application Standards. In a 
self-assessment of their Technology Proficiency across three time points, immersion teachers 
considered themselves to be increasingly more technology literate than control teachers in areas 
involving technology operations (e.g., sending email and using software applications) and pedagogical 
skills (e.g., creating electronic presentations and creating lessons plans integrating technology). 
Estimated yearly growth trajectories for immersion and control teachers in schools with average levels 
of student poverty were 0.35 and 0.18 scale-score points per year, respectively, on a 7-point scale. 
Similarly, teachers in immersion schools used technology significantly more often for administrative 
and classroom management puiposes. Estimated yearly growth trajectories for immersion and control 
teachers relative to their Professional Productivity were 0.26 and 0.09 scale-score points per year, 
respectively, on a 5-point scale. Teachers in immersion schools used technology more frequently for 
purposes such as communicating with students, posting information on a website, administering an 
online assessment, and accessing model lesson plans. 

Teachers in immersion schools expressed stronger ideological associations across time with 
technology integration and learner-centered practices. Initially, immersion and control teachers 
expressed similar views on instructional practices involving technology; however, immersion teachers 
changed their instructional beliefs at a significantly more positive rate. For Technology Integration, 
the mean estimated growth trajectory for immersion teachers in schools with average poverty was 0.75 
scale points per year compared to 0.27 for control teachers (on a 7-point scale). Thus, immersion 
teachers indicated that they increasingly employed actions supporting curricular and instructional 
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infusion of technology, such as promoting students’ authentic problem solving or critical thinking 
through technology. Immersion teachers also expressed increasingly stronger affiliations with 
constructivist or learner-centered practices, such as having students establish individual learning goals 
and emphasizing experiential learning. The estimated yearly growth in learner-centered practices for 
immersion and control teachers in schools with average poverty were 0.48 and 0.20 scale-score points, 
respectively, on a 7-point scale. 

Teachers at schools with higher concentrations of student poverty grew in technology 
proficiency and adopted new ideologies at slower rates. Teachers who taught at schools with higher 
levels of student poverty grew in technology proficiency at significantly slower rates than their peers 
in more advantaged schools. As the level of school poverty increased, the proficiency gap between 
teachers widened. Similarly, teachers in schools with higher student poverty embraced technology 
integration and learner-centered practices at slower rates. Weaker supports for implementation at 
higher poverty immersion schools may at least partially explain teachers’ slower progress. Core- 
subject teachers in these schools reported considerably fewer technology-related professional 
development hours than teachers in more advantaged schools (34 hours over two years, on average, vs. 
52 hours). And, teachers’ perception of the robustness of their professional development was 
positively correlated with the strength of their affiliation with new ideologies. The characteristics of 
teachers employed at schools also may contribute to differences. Higher poverty immersion schools 
had proportionately more male teachers. Accordingly, our analyses indicated that male teachers in 
middle schools were significantly less likely than their female counterparts to embrace innovative 
methods (i.e., technology integration and learner-centered instruction). 

Given greater abundance of technology, teachers in immersion schools collaborated more often 
with their peers on technology-related issues than control teachers, and students used technology 
more often in immersion classrooms. With increased availability of technology, teachers at 
immersion schools compared to control had a significantly steeper growth trend for collaborative 
interactions with colleagues that supported improvements in instructional practices involving 
technology (e.g., developing lesson plans or exchanging information about students). Also, immersion 
teachers had a significantly steeper growth rate than control teachers for the frequency of their 
students’ Classroom Activities involving technology (0.29 scale-score points per year vs. 0.04 points 
on a 5-point scale for teachers in schools with average poverty). Despite their positive growth trend, 
statistics indicated that by spring 2006, teachers in immersion schools had students use various 
technology resources infrequently (i.e., about once or twice a month). While the overall level of 
technology use was low, there was substantial teacher-to-teacher valuation in students’ classroom 
activities; mathematics teachers used technology considerably less often than English language aits, 
science, and social studies teachers. 

Availability of technology resources had little, if any, effect on the intellectual challenge of 
immersion teachers’ lessons. Technology immersion’s theorized impact on student achievement 
hinges on technology’s facilitation of more rigorous and authentic learning experiences. New 
resources are expected to promote students’ high-level thinking, concept formation, inquiry and 
investigation, information utilization, exposure to places beyond the classroom, and real-world 
learning (Bransford et al., 2003; Goldman, Cole, & Syer, 1999; Johnston & Cooley, 2001; Sulla, 

1999). These kinds of experiences are important because of established links between more 
challenging and authentic pedagogy and academic achievement (e.g., Newman & Associates, 1996; 
Newmann, Bryk, & Nagoaka, 2001). To gauge progress toward expected technology use, researchers 
rated the Intellectual Challenge of teachers’ observed lessons (Newmann, Secada, & Wehlage, 1995). 

Observations of core-subject teachers (English language aits, mathematics, science, and social studies) 
in fall 2004 and spring of 2005 and 2006 revealed no statistically significant differences between the 
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intellectual demand of immersion and control teachers’ lessons. Across all observed classrooms, 
lessons generally failed to intellectually challenge students, with ratings usually below 2 on the 5-point 
scale. Observational data for sixth-grade classrooms showed that lessons in immersion and control 
classrooms most often focused on student acquisition of facts, definitions, and algorithms (44% and 
41% of observed time, respectively) and less often centered on writing communication related to the 
lesson (7% and 4% of time), constructing knowledge (10% and 13% of time), or engaging in 
disciplined inquiry (7% and 5% of time). 

Effects of Immersion on Students and Learning 

We examined the effects of immersion on students and their learning by measuring mediating 
variables across three time periods for Cohort 1 students (fall 2004, spring 2005, and spring 2006) and 
two periods for Cohort 1 students (fall 2005 and spring 2006). Analyses for Cohort 1 included 
approximately 3,100 students (1,454 immersion and 1,646 control); analyses for Cohort 2 included 
about 4,033 students (1,988 immersion and 2,045 control). 

Technology immersion significantly increased students’ technology proficiency and narrowed 
the gap between economically advantaged and disadvantaged students. Students in immersion 
schools reported significantly higher levels of technology proficiency than control students. Thus, 
immersion students made greater progress toward mastery of the Texas Technology Applications 
standards (e.g., sending an email attachment, creating a presentation, managing documents, using 
spreadsheets, and keeping track of websites). Estimated yearly growth in proficiency for economically 
advantaged and disadvantaged immersion students in Cohort 1 (0.41 and 0.47 scale-score points, 
respectively, on a 5-point scale) were nearly twice the rates for their control-group counterparts (0.21 
and 0.27 scale-score points). Consequently, by the end of seventh grade, economically disadvantaged 
students in immersion schools surpassed advantaged control students in proficiency. Similarly, for 
Cohort 2, sixth graders, immersion had a significantly positive effect on students’ technology 
proficiency (Effect size [ES] = 0.30). Immersion students’ enhanced technical skills apparently 
allowed them to deal with technical troubles that typically accompany increased technology access and 
use because the extent of technical problems reported by immersion and control students did not differ 
by statistically significant margins. 

Students in immersion schools used technology significantly more often in core-subject 
classrooms and interacted more frequently with their peers in small groups. Similar to their 
teachers’ reports, Cohort 1 students at immersion schools had a significantly steeper growth trend for 
the frequency of Classroom Activities involving technology than control students. The yearly rates of 
change in class activities for economically advantaged and disadvantaged immersion students were 
0.35 and 0.41 scale-score points, respectively (on a 5-point scale), compared to 0.03 and 0.09 points 
for comparable control-group students. Results for Cohort 2 students, similarly, revealed significant 
and practically important differences in Classroom Activities favoring immersion schools (ES = 0.83). 
Along with greater uses of classroom technology, students in immersion schools also had more 
frequent opportunities for collaborative and active learning. Growth rate coefficients for Cohort 1 
showed that students in immersion schools had increasing opportunities across time for small-group 
work with their peers, whereas control students reported less frequent small-group activities as they 
advanced to seventh grade. The greater frequency of small-group work in immersion schools was 
replicated for Cohort 2 students (ES = 0.25). 

Despite significant increases in classroom technology use in immersion schools, mean use statistics 
indicated that students used technology resources infrequently (about once or twice a month). 
Moreover, classroom observations showed that immersion students often used technology in rather 
conventional ways. Observed students most frequently used a word processor for writing, learned and 


79 



practiced skills (typically multi-choice exercises or digitized worksheets), created or made 
presentations (using PowerPoint or Keynote), or conducted Internet searches for information on an 
assigned topic. Students had little chance to use technology for activities that supported deeper 
thinking and understanding, such as using technology to analyze or manage data, communicate with 
peers and experts, enhance understanding through simulations or modeling, or to visually represent or 
investigate concepts. Thus, changes in classroom activities and organizational structures in immersion 
classrooms did not necessarily alter the rigor or relevance of students’ experiences with core-subject 
content. 

Technology immersion had no significant effect on student self-directed learning. We theorized 
that the opportunities for independent and self-guided learning afforded through one-to-one 
technology would positively affect students’ personal self-direction. Accordingly, student cohorts 
completed the Style of Learning Inventory as a measure of their self-direction. Findings in the second 
year replicated first-year results showing there was no significant immersion effect on students’ self- 
direction. As both immersion and control students in Cohort 1 progressed from sixth to seventh grade, 
their responses to statements measuring self-direction revealed significantly negative growth trends. 
Thus, these students reported less self -regulated learning behaviors across time. Results for Cohort 2, 
sixth graders, likewise, revealed no significant immersion effect on student self-direction (ES = 0.03). 

Outcomes for measures of student engagement varied. Students in immersion schools had 
significantly fewer disciplinary actions, similar levels of school satisfaction, and significantly 
lower school attendance rates than control-group students. A frequently cited benefit of one-to- 
one computing is increased student engagement as measured by indicators such as stronger 
commitment to academic work, increased attendance, and reduced discipline problems (e.g., MEPRI, 
2003; Lowther, Ross, & Morrison, 2003; Rockman ET AL., 1998; 1999; Russell, Bebell, & Higgins, 
n.d.). Similarly, during site visits for this study, interviewed administrators, teachers, and students 
cited greater student interest and motivation for school and learning as positive immersion effects. For 
our quantitative measures, however, immersion students exhibited significantly stronger school 
engagement through more positive behavior, but they did not express greater satisfaction with school 
and they did not attend school more regularly than control students. 

Behavior and discipline problems. Disciplinary Action Reports submitted to the TEA for each 
student during the 2005-06 school year, similar to the prior year, showed that immersion students had 
proportionately fewer behavioral and disciplinary problems than their counterparts in control schools. 
Cohort 1 immersion and control students had an average of 0.76 and 0.90 disciplinary actions, 
respectively (ES = 0.14); Cohort 2 immersion and control students had an average of 0.53 and 0.69 
disciplinary actions, respectively (ES = 0.16). Even though effect sizes for the mean differences are 
small, having fewer disciplinary actions per student in middle schools may have important practical 
benefits in terms of day-to-day personnel time and effort required for addressing discipline problems 
that remove students from classrooms. 

School satisfaction. Contrary to first-year results showing higher levels of school satisfaction for 
Cohort 1 students, survey outcomes for the second year showed no significant difference in school 
satisfaction between students in immersion and control schools. Students across both comparison 
groups and cohorts expressed correspondingly modest levels of satisfaction with the kinds of work 
they do in classes and with the relevance of their schoolwork. The lower satisfaction level for 
Cohort 1 , seventh graders in immersion schools may reflect a drop in the initial euphoria that students 
experienced upon receiving their laptops in the first year. Additionally, as schools experienced 
disciplinary and safety problems involving students’ use of email and the Internet, they restricted 
students’ Internet access and use of laptops for communication puiposes in the second year. Lower 
school satisfaction may reflect immersion students’ disenchantment with stricter laptop policies. 
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School attendance. Unexpectedly, students in immersion schools had significantly lower school 
attendance rates than control-group students. For Cohort 1, average school attendance rates decreased 
as students advanced from fifth to seventh grade. The yearly estimated decline in attendance for 
economically advantaged immersion students (-0.57 percentage points) was greater than the annual 
change for similar control students (-0.36 points). Thus, at the end of seventh grade, advantaged 
students in immersion schools had an estimated average attendance rate of 95.9% compared to 96.4% 
for their control-group counterparts. Attendance rates for economically disadvantaged students 
decreased at an even faster pace (coefficients of -0.73 and -0.51 percentage points for immersion and 
control students, respectively). Results for Cohort 2 students, similarly, showed small but statistically 
significant differences in attendance rates favoring students in control schools (ES = 0.07). 

Researchers currently have no evidence that explains why immersion students have lower attendance 
rates; however, we will continue the investigation of attendance trends in the third evaluation year. 

Effects of Immersion on Academic Achievement 

The ultimate goal of technology immersion is increasing middle school students’ achievement in core 
academic subjects as measured by state assessments. For analyses reported below, students’ TAKS 
scale scores were standardized and then normalized as T scores with a mean of 50 and a standard 
deviation of 10. Analyses for Cohort 1 included about 3,450 students (1,590 immersion and 1,860 
control); Cohort 2 included about 3,838 students (1,805 immersion and 2,033 control). 

Technology immersion had no statistically significant effect on Cohort 1, seventh graders’ 
achievement in reading, mathematics, or writing. For Cohort 1 students, we used three-level 
hierarchical linear modeling (F1FM) to estimate mean rates of change in TAKS reading and 
mathematics scores and a two-level F1FM model to estimate the effects of immersion on writing 
scores. 

• Reading. Controlling for student and school poverty, there was no significant effect of 
immersion on students’ growth rate for TAKS reading. Advantaged students in control 
schools (with average poverty) began with a mean TAKS reading T score of 53.83, and their 
achievement decreased by about 0.55 T-score point per year (coefficient of -0.547). Estimated 
reading scores for advantaged students in immersion schools (with average poverty) decreased 
at a slightly slower yearly rate than control students (positive coefficient of 0.257). 
Economically disadvantaged students in both immersion and control schools grew in reading 
achievement at a significantly faster rate than their more advantaged peers (0.461 T-score 
points per year). Combined with the immersion result, this yielded a positive boost in reading 
achievement for economically disadvantaged immersion students. 

• Mathematics. After controls for student and school poverty, there was no significant effect of 
immersion on students’ growth rate for TAKS mathematics. Advantaged students in control 
schools (with average poverty) had an estimated initial mean T score of 52.81, and their math 
scores decreased by about 0.32 T- score points per year (coefficient of -0.315). Estimated math 
scores for advantaged immersion students decreased at a slightly slower rate (positive 
coefficient of 0.222). In contrast to reading, economically disadvantaged students at both 
immersion and control schools grew in mathematics achievement at a significantly slower rate 
than their more advantaged peers (-0.285 T-score points per year). 

• Writing. Cohort 1 students completed TAKS writing assessments in 2003 (4th grade) and 
again in 2006 (7th grade). After adjusting for initial achievement, student demographic 
characteristics, and school poverty, there was no statistically significant difference in the 2006 
writing scores for students in immersion and control schools. The immersion effect was 


81 



negative but not by a statistically significant margin (about 0.91 T-score points lower than the 
control group). 

Technology immersion had no statistically significant effect on Cohort 2, sixth graders’ reading 
achievement; however, immersion had a significantly positive effect on mathematics scores for 
higher achieving students. We analyzed the effects of immersion on Cohort 2 students’ TAKS 
reading and mathematics scores using two-level HLM models. Cohort 2 students completed TAKS 
assessments in 2005 (5th grade) and again in 2006 (6th grade). 

• Reading. Controlling for students’ prior achievement, demographic characteristics, and school 
poverty, there was no significant difference in the 2006 TAKS reading scores for students in 
immersion and control schools. The immersion effect on reading was positive (about 0.24 T- 
score points) but not by a statistically significant margin. 

• Mathematics. After controls for students’ prior achievement, demographic characteristics, 
and school poverty, there was no overall significant difference between immersion and control 
students’ TAKS mathematics scores. The overall immersion coefficient was positive (0.95 
T-score points) but not by a statistically significant margin. However, there was a statistically 
significant immersion effect on mathematics achievement that acted through students’ pretest 
scores. Other factors being equal, having higher pretest scores predicted larger gaps in 2006 
math scores favoring immersion students. Thus, immersion had a significantly positive effect 
on mathematics achievement for higher achieving sixth graders. 

Second-year achievement trends generally favored technology immersion schools. Although 
TAKS scores for students attending immersion and control schools usually did not differ in the second 
year - by statistically significant margins, noteworthy achievement trends emerged. In the first project 
year, TAKS reading and mathematics achievement trends favored control schools. Conversely, in the 
second year, immersion schools had more positive achievement trends than control schools across 
both student cohorts and for both reading and mathematics. In particular, Cohort 1, seventh graders 
had more positive reading and math growth trajectories than control-group students. Similarly, reading 
and mathematics outcomes for Cohort 2, sixth graders showed positive achievement tendencies 
favoring immersion students. 

Outcomes for TAKS writing favored students in control schools. The assessment of writing, however, 
differed from other subject areas in several ways. First, the span of time between the pretest (2003) 
and posttest (2006) was wider. The testing mode for writing also may be a confounding factor. The 
TAKS assessment was administered in traditional paper-and-pencil format, and some research shows 
that paper-and-pencil assessments underestimate the writing achievement of students who regularly 
use word processors for writing (e.g., Russell & Haney, 1997). Other research, however, has found no 
testing mode differences in aggregate writing performance, although outcomes for individuals may 
vary (e.g., Sandene et al., 2005). 

Second-year findings provide formative evaluation outcomes. The evaluation of technology 
immersion is a four-year, longitudinal study, and findings from the second year reflect preliminary 
outcomes. In designing the study, we thought that some effects might emerge during the early 
implementation years, but we also believed that changes in longer term outcomes, such as student 
achievement, might require at least three years to surface (i.e., time for Cohort 1 students to progress 
from sixth to eighth grade). Specifically, we have noted that low project implementation at schools 
during the first two years may diminish immersion’s potential impact on achievement. Major concerns 
include students’ inconsistent use of laptops across classrooms and subject areas, uneven provision of 
professional development supporting the design of effective technology-infused lessons, and 
variability in students’ access to laptops both within and outside of school. Additionally, outcomes so 


82 



far have focused mainly on TAKS reading and mathematics. In the third year, Cohort 1 eighth graders 
will complete TAKS social studies and science assessments, so outcomes will be available for each of 
the core-subject areas. 

Moreover, although student achievement outcomes as measured by TAKS scores are extremely 
important, there are other outcomes for immersion students that may contribute to their long-term 
success. Certainly, technology immersion has narrowed the technology equity gap for economically 
disadvantaged students. Many students who previously had no technology in their homes are 
becoming computer literate through their experiences with laptops. Administrators, teachers, and 
students alike at immersion schools believe that these middle school students are better prepared for 
future educational and workforce requirements and for 21 st Century expectations, such as 
communication skills, and information and media literacy. 

In the section to follow, we describe how the varying levels of implementation may have contributed 
to second-year results. 

Nature of Second-Year Implementation 

Most of the middle schools struggled in the second year to implement the prescribed components 
of technology immersion. Full implementation of the immersion model requires support in several 
ways: Leadership, Teacher Support (buy-in), Parent and Community Support, Technical Support, and 
Professional Development. Given adequate supports, teachers are expected to reach high levels of 
Classroom Immersion, and Student Access and Use of technology is expected to be robust. The 
Implementation Index, a composite campus z score that measures the overall presence of immersion 
components, showed that about a third of middle schools (6 of 21) attained a stronger presence of the 
technology immersion components that more nearly approximated expected standards, whereas two- 
thirds had lower implementation levels. Overall, mean immersion standard scores (ranging from 2.48 
to 3.06) showed that supports for technology immersion from school principals, teachers, parents and 
community members, technical staff, and professional development providers generally failed to meet 
full implementation standards (3.50 to 4.00). 

Given generally low-to-moderate supports for immersion, the overall extent of Classroom Immersion 
(2.48) and the robustness of Student Access and Use (2.17) were below expectations. Overall, 
immersion teachers’ behaviors in the second project year were consistent with other research on 
teachers’ adoption of technology-based practices. Teachers tended to use new resources to support 
their existing teaching practices and they used technology for student learning to a limited extent 
(Sanholdtz, Ringstaff, & Dwyer, 1997; Windschitl & Sabi, 2002). Although many teachers have made 
significant strides in using laptops and digital resources in their classrooms over two project years, 
most teachers still need time and continuing support for incremental classroom change. 

The strength of professional development and other supports were associated with higher levels 
of classroom and student immersion. Variability in the extent and quality of each school’s 
professional development component was a major obstacle to teachers’ growth in creating technology- 
immersed classrooms. Even though the immersion model required that a quarter of grant funds be 
expended for professional development that advanced technology integration into teaching and 
learning, the professional development design rested largely with individual districts and campuses 
and their selected technology vendors (mainly Apple or Dell). An analysis of associations between the 
various immersion support components and classroom practices indicated that strength of campus 
professional development was significantly correlated with teachers’ reported levels of classroom 
immersion. Robust professional development was strongly associated with teachers’ ideological 
beliefs about the adoption of technology integration and learner-centered practices. 
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Leadership for immersion also emerged as an important factor in advancing school change. Teachers’ 
perception of leadership strength was significantly associated with their commitment to technology 
innovation, views on parent and community support, perception of professional development quality, 
and opinions about technical support adequacy. Thus, principals appeared to influence teachers’ 
attitudes toward technology through their provision of supports for changed practice. Similarly, 
students’ access to and use of technology for learning was significantly related to their teachers’ 
greater involvement in professional development as well as the strength of the school’s supports for 
immersion. 

A continuing challenge in the second year was the consistent provision of laptops for students 
both within and outside of school. Student laptop access varied widely both across and within 
schools. The average number of laptop access days reported by students ranged from 42 to 178 days, 
with about a tenth of campuses having full access (the targeted 170 to 180 days per student). Although 
students in a fully immersed school should have access to their wireless laptops and resources nearly 
the entire school year, we found that student access was limited in the second year by a number of 
factors, such as disciplinary infractions, technical issues, time for repairs, and in a few cases, parent 
resistance. Additionally, some immersion schools allowed students to have unlimited access to laptops 
outside of the school day, while others restricted students’ out-of-school access to a series of days or to 
laptop check-outs for teacher-assigned schoolwork. Overall, laptops’ potential influence on learning 
varied across students and schools. The school’s emphasis on professional development and the level 
of teacher and parent/community support for immersion were positively associated with students’ 
laptop use for learning at home (i.e., for homework in core subjects or learning games). 

Schools with a greater proportion of economically disadvantaged students had lower levels of 
implementation. Schools with more economically disadvantaged student populations had 
significantly lower levels of implementation. Accordingly, teachers at these schools grew in 
proficiency and created immersed classrooms at significantly slower rates than teachers in more 
advantaged schools. Evidence suggests that schools serving mainly disadvantaged and often low- 
performing student populations face special challenges in implementing technology immersion, a 
whole-school initiative that involves profound school and classroom change. Such schools may need 
additional time to prepare for immersion and additional supports to devise and enact implementation 
plans that meet the specific needs of teachers, students, and parents at these schools. 

Researchers have conducted additional analyses to explore associations between the level of 
implementation and student academic achievement. However, these analyses arc complex due to 
varying student attrition rates across immersion schools and have produced inconsistent results. We 
will continue in the third evaluation year to explore these associations. 
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Appendix A 

Theoretical Framework for Technology Immersion — Literature Review 


The theoretical framework (Figure 1.1) guides the evaluation. The research literature underpinning the 
framework is provided in sections to follow for school, teacher, and student valuables. In some cases, 
sources relate specifically to educational technology, whereas in other instances, evidence comes from 
studies of education in general. Research evidence for some valuables is relatively robust; in other 
areas, evidence is weaker. Although research on one-to-one computing initiatives has grown in recent 
years, there are still few experimental studies or studies with well matched comparison groups that 
provide evidence of causal effects. 

School-Level Variables 

In a “technology immersed” school, technology resources are ingrained in the school’s organizational 
and cultural environment. Technology immersion, therefore, should change not just classroom 
instruction and learning, but also the nature of interactions between student and teacher, teacher and 
teacher, teacher and principal, and the school within the surrounding community (Dwyer, 1994). 
Considering the systemic nature of technology immersion, the evaluation examines factors that help to 
explain how and under what conditions technology affects students’ learning opportunities and 
academic achievement. The sections below describe the key valuables of interest at the school level, 
including leadership, innovative culture, parent and community support, and technical support. 

Leadership 

Over the past several decades, researchers have concluded consistently that school leadership is 
critical in developing and maintaining conditions that support school change and academic 
improvement (e.g., Hallinger & Fleck, 1996 cited in Spillane, 2003; Leithwood, Seashore, Anderson, 

& Wahlstrom, 2004). Similarly, administrative support is a major factor that influences technology 
integration (International Society for Technology in Education, 2002; Bradburn & Osborne, 2007). 
Leaders in a technology-enhanced environment must be “champions of technology, teaching, learning, 
and students” (Johnston and Cooley, 2000, p. 95). The principal, in particular, is a pivotal figure in 
effective technology implementation. The visionary principal is one who sees the integral relationship 
between technology and education, and marshals resources to help teachers master effective practices 
(Tinucci, 2000). Additionally, effective principals are “transformational leaders” who create more 
collaborative teaching and learning environments through their facilitation of opportunities for 
technology specialists and teachers to share their knowledge, experiences, and insights (Bradburn & 
Osborne, 2007). 

A consistent vision and plan for change is also essential for whole-school reform efforts such as 
technology immersion. Shared vision, or buy-in, moves schools toward substantive changes in 
instructional approaches and improved student outcomes (Leithwood et al., 2004). Conversely, 
without broad-based support, technology immersion may be untapped resource that has little impact 
on student learning (Cradler, 1992; Means & Olson, 1994). 

Innovative Culture 

The school culture may either promote or impede whole-school initiatives such as technology 
immersion. When undertaking innovation, the organization’s shared commitment to change and ability 
to build capacity for doing things in a new way are important (Senge, 1999). In education, some 
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schools are more successful than others in enacting and sustaining innovation, and in more effective 
schools, changed practice is a collective rather than an individual enterprise (Fullan, 1993). Similarly, 
movement towards new ways of teaching and learning with technology is more significant if teachers 
are able to work collaboratively (Chapman, 1996). Shared professional learning opportunities provide 
a viable means to stimulate innovative teaching practices (Birman, Desimone, Porter, & Garet, 2000; 
Dibbon, 2003). Considering prior research, we believe that educators’ collective experiences at 
immersion campuses will advance their shared understanding of technology’s use and encourage 
integration efforts. Schools that begin the project with more collaborative cultures may advance at a 
faster pace (Fullan, 1999). 

Parent and Community Support 

The local community also may influence technology immersion. Its constituents consist of parents, 
neighborhood residents, local professionals, and elected school board officials. Educating and 
involving the community has been identified as a key component in ensuring successful change in 
educational practices (Desimone, 2002; Goertz, Floden, & O’Day, 1996; Leithwood et al., 2004). If 
parents and community members are “on the same page” as the school with regal'd to technology 
immersion, they can contribute the kind of supports and resources required for changes in educational 
practices. At immersion campuses, community outreach may take many forms, such as participation 
on a technology committee, attendance at informational sessions or workshops, the dissemination of 
information through district and campus websites, or media releases to spread the word about 
technology immersion. Most important, in a one-to-one computing project, parents must be partners in 
assuming responsibility for the appropriate use of laptops outside of the school. 

Technical Support 

Texas has strongly supported the infusion of technology into its schools (Texas Education Agency, 
2002; 2006). Consequently, at the start of this project, both treatment and control campuses had 
existing inventories of technology hardware, software, and educational programs. Districts and 
campuses also had human resources such as technology coordinators and technical support personnel 
who supported technology at the district and campus levels. Given existing contextual conditions, and 
the infusion of resources through technology immersion, an examination of the nature and quality of 
technical support at participating schools is important. 

Teacher Variables 

At the teacher level, we theorize that technology immersion leads to increased technology proficiency, 
greater use of technology for professional productivity, more frequent opportunities for students to use 
technology in classrooms, and pedagogical changes such as increased technology integration and more 
learner-centered instruction. New technology also is expected to advance the intellectual demands of 
lessons and assignments. Moreover, teachers in schools that are immersed in technology should begin 
to collaborate more often with their peers as they experiment with new instructional technologies and 
digital resources. 

Technology Proficiency 

A number of studies associate teachers’ technology proficiencies with technology implementation. 
Research indicates that teachers need a solid foundation of technology literacy before they can 
successfully integrate technology into the curriculum. Teachers must learn to use technology 
comfortably and efficiently (Dusick, 1998-1999; Goldsworthy, 2000). Studies also show that teachers 
with stronger computer skills use technology in a greater number of ways and on a more regular basis, 
and these teachers are more likely to increase their technology-use frequency over time (Ronnkvist, 
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Dexter, & Anderson, 2000). Moreover, teachers with the strongest technology proficiencies use 
technology in more innovative ways in their content areas (Becker, 2000). 

Unfortunately, research indicates that many teachers lack the proficiencies and understanding 
necessary to apply technology resources to instruction and learning effectively. A national study found 
that more than half of teachers felt only somewhat prepared to use technology for instruction, and 
more experienced teachers felt less prepared than their more novice counterparts (Smerdon et al., 
2000). Surveys of Texas teachers have revealed improvements in proficiencies across time, but 
teachers’ proficiency levels remained below targeted standards (Shapley, Benner, Heikes, & Pieper, 

2002) . Similarly, 2005-06 statewide outcomes for the Texas Teacher STaR Chart (a measure of 
teachers’ technology readiness) showed that nearly three of four Texas teachers rated their progress 
relative to the Teaching and Learning area as either Early Tech (14.7%) or Developing Tech (55.6%). 
Only one in four teachers believed they had attained proficiencies designated as Advanced Tech 
(23.7%) or Targeted Tech (5.8%) (Texas Region 10 Education Service Center & Texas Education 
Agency, 2006). 

Professional Productivity 

Skilled teachers also are more likely to use technology as a tool to enhance their own professional 
productivity, including actions such as communicating with students and parents by email, creating 
electronic lesson plans, or accessing information from the Internet for lessons (Shapley et al., 2002). 
Researchers typically have not investigated teachers’ use of technology for professional productivity, 
but it is important in Texas because state standards call for teachers to use technology for 
communicating effectively, as well as for acquiring, analyzing, and evaluating a variety of electronic 
information. In an immersed school, teachers are expected to increasingly communicate by email, 
report attendance and submit lesson plans electronically, post information on a class or campus 
website, and analyze and interpret electronic data from assessments. 

Classroom Technology Use 

The link between increased technology access and increased classroom use is well documented (see 
Chapter 1). Teachers use computers and the Internet more often when technologies are available in 
their classrooms rather than in other locations in the school (Becker, 2001; Smerdon et al., 2000). 
Teachers involved in Maine’s one-to-one initiative, in fact, used technology more often, possessed a 
broad knowledge of technology resources, and made progress in incorporating technology into 
practice (MEPRI, 2004). Thus, we assume that providing laptops for each student in an immersed 
school will increase students’ opportunities for classroom technology use. 

Technology Integration and Learner-Centered Instruction 

Abundant technology hardware and software is important, but if those resources are not well 
integrated into instructional approaches and learning experiences, the impact on student achievement 
may be negligible. Notably, studies show that teachers’ ideologies affect the likelihood of technology 
integration, with teachers’ perceived costs and benefits influencing changed practices (Zhao & Frank, 

2003) . Research also suggests that teachers’ understanding of new learning theories and understanding 
of how technology supports enriched learning opportunities are important (Bransford, Brown, & 
Cocking, 2003; Johnston & Cooley, 2001). Researchers studying the Apple Classrooms of Tomorrow 
(ACOT) found that abundant access to classroom technology changed teachers’ beliefs as well as their 
instructional approach. Teachers’ beliefs and practices evolved along a technology integration 
continuum that gradually led to effective instructional practices. Movement from the entry phase to 
invention (technology-intensive environments) required time and ongoing support (Dwyer, Ringstaff, 
& Sandholtz, 1991). 
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Specifically, researchers found that ACOT teachers began to incorporate more collaborative work and 
fewer teacher-centered, lecture-oriented lessons in favor of student-centered ones (Baker, Gearhart, & 
Herman, 1994). Subsequent studies, likewise, have found evidence of teachers adjusting their 
pedagogical style, with students taking more responsibility for their own learning in one-to-one laptop 
classrooms (MEPRI, 2003), and classroom structures that shifted from large group to students working 
independently or to more student-centered activities (Rockman ET AL., 1998; Russell, Bebell, Cowan, 
& Corbelli, 2002). Other evidence, however, suggests that some teachers view technology as an add- 
on or reward for students who finish their seatwork rather than an integral part of their pedagogical 
repertoire (Rockman ET AL., 1998). 

Intellectual Challenge 

Technology immersion’s main benefit may stem from opportunities for more complex modes of 
teaching and learning. Research on technology-infused classrooms reveals positive attributes, such as 
the ability to bring real-life problems into the classroom or high-quality simulations of them. 
Technology also allows teachers to model thinking strategies and allows individual learners to 
approach tasks in different ways using different learning strategics (Goldman, Cole, & Syer, 1999; 
Many, Fyfe, Lewis, & Mitchell, 1996; Sulla, 1999; Temple & Rodero, 1995). This view of 
technology’s potential for more advanced learning contrasts with evidence on prevailing classroom 
conditions. While three-quarters of teachers nationally report using computers or the Internet for 
instruction, most lessons fail to involve complex inquiries, explorations, or problem-solving activities 
(Doherty & Orlofsky, 2001). Similarly, Texas students and teachers use technology mainly at a basic 
level, with technology used most often for tasks such as conducting Internet research on an assigned 
topic (Shapley et al., 2002). 

Collaboration 

Research suggests that teachers need time to discuss technology use with other teachers. Professional 
collaboration includes communicating with educators in similar situations and with teachers who have 
previous technology experiences. Collaboration may occur in face-to-face meetings or through 
technology venues such as email or videoconferencing. Teachers in the Maine laptop initiative, for 
example, believed their most effective professional development activity was informal help from 
colleagues. E-mail, listservs, and websites enabled Maine teachers to exchange information and stay in 
touch with their peers (MEPRI, 2003). Moreover, Zhao and Frank report that “teachers who perceived 
pressure from colleagues were more likely to use computers for their own puiposes, and teachers who 
received help from colleagues were more likely to use computers with their students” (2003, p. 825). 

Student Variables 

Over the past decade, a growing body of research points to positive effects of technology on students’ 
skills, learning, and achievement. In the research literature, evidence suggests that technology access 
fosters positive student effects for technology use, technical proficiencies, motivation and engagement, 
intellectually challenging schoolwork, self-direction, and to a lesser extent, academic achievement. 

Technology Use 

Technology is used more often for instructional and learning puiposes in one-to-one laptop classrooms 
(Russell, Bebell, & Higgins, n.d.). Additionally, students involved in ubiquitous technology initiatives 
use technology more often outside of school. Russell et al. (n.d.) found that students in one-to-one 
classrooms used computers at home more frequently for academic puiposes. Likewise, other 
researchers found that students spent less time watching television and more time on homework after 
they received laptop computers (Baldwin, 1999). Moreover, laptops provided a means of “closing the 


98 



digital divide” between more advantaged students who had access to computers and the Internet at 
home and those without technology outside of school (Rockman, 2003). 

Technology Proficiency 

Students’ technology proficiencies reportedly increase with ubiquitous technology. Laptop students in 
one study considered themselves more proficient users of Word, Excel, PowerPoint, the Internet, 
email, and CD-ROMS than non-laptop students (Rockman ET AL., 1998). Similarly, fifth and sixth 
graders who received laptop computers in another study reported increased computer skills and better 
Internet research capabilities (Lowther, Ross, & Morrison, 2001). In another study, German high 
school students with laptops made greater gains than comparison students on measures of technology 
literacy, such as knowledge of hardware and the operating system, productivity tools, and Internet use 
(Schaumburg, 2001). 

Motivation and Engagement 

Numerous studies report links between one-to-one technology and increased student engagement 
(MEPRI, 2003; Rockman ET AL., 1998; Russell et al., n.d.; Woodul, Vitale, & Scott, 2000). The five- 
year ACOT evaluation established a link between technology use and student attitudes. Students 
voluntarily used time outside of school to work on technology-based projects, and they often initiated 
their own computer-related projects (Baker, Gearhart, & Herman, 1994). Students involved in the 
Maine Learning Technology Initiative, similarly, found school and learning more interesting and 
preferred using laptops for most school-related tasks (MEPRI, 2003). 

Additionally, studies have examined the relationship between technology and student behavior. In a 
statewide study in Florida, middle schools experienced fewer student conduct violations and 
disciplinary actions as the number of computers in use per student increased (Barron, Hogarty, 
Kromery, & Lenkway, 1999). Other studies, likewise, report decreased discipline problems associated 
with one-to-one computing (Baldwin, 1999; MEPRI, 2003). In another study, a computerized 
curriculum positively affected the psychosocial and academic outcomes of students identified as 
chronically disruptive (Aeby, Powell, & Carpenter- Achy, 1999-2000). 

An evaluation of the North Carolina Laptop Notebook Project revealed a strong correlation between 
computer use and improved school attendance. Students participating in the laptop program had fewer 
absences and late arrivals as compared to non-participants (Stevenson, 1998). In Henrico County 
Public Schools in Virginia, preliminary evidence linked increased student motivation, engagement, 
and interest to one-to-one computing (Zucker & McGee, 2005). 

Intellectual Work 

Existing studies suggest that student technology use most commonly involves productivity tools, 
Internet research, and drill and practice activities. Activities involving higher-order thinking and peer 
collaboration, such as technology-based projects, multimedia authoring, problem solving with 
spreadsheets or databases, or correspondence with experts, are less common (Becker, 1999, 2001; 
Denton, Davis, & Strader, 2001; Smerdon et al., 2000). Contrary to prevalent practice, some believe 
that technology, at its best, can “facilitate deep exploration and integration of information, high-level 
thinking, and profound engagement by allowing students to design, explore, experiment, access 
information, and model complex phenomena” (Goldman et al., 1999). Additionally, technology allows 
students increased access to and use of a wide range of information, facilitating greater inquiry and 
investigation, exposure to places and resources beyond the classroom, and development of a stronger 
knowledge base (CEO Forum, 2001; Johnston & Cooley, 2001). 
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New circumstances and opportunities — not technology on its own — can impact student achievement. 
Several studies have established tentative links between interactive technologies and higher level 
reasoning and problem solving (Baker et al., 1994; Hopson, Simms, & Knezek, 2002). New 
technologies, apparently, allow students to build knowledge by doing, receiving feedback, and 
continually refining their understanding (Barron et al., 1999; Bereiter & Scardamalia, 1993). 
Technology also provides a medium for bringing real-world problems into the classroom for students 
to explore and solve. Students involved in the Jasper Woodbury Problem Solving Series, for example, 
had positive gains in mathematical problem solving, communication abilities, and attitudes toward 
mathematics (e.g., Cognition and Technology Group at Vanderbilt, 1997). 

Self-Directed Learning 

Several studies associate technology use with increased student self-directed learning. The connection 
assumes that working one-to-one with technology allows students to have hands-on, self-directed 
experiences since they work independently much of the time. The theory of self-regulation posits that 
a learner who knows how to be self-directed and independent will be more successful than one who is 
highly dependent on structured guidance (Zimmerman, 1989). The teacher’s role is to scaffold 
learning by making thinking processes more tangible and by modeling learning strategies (Bolhuis, 
1996; Corno, 1992; Leal, 1993). Since self-directed learners are responsible owners and managers of 
their own learning process, control shifts over time from teachers to learners (Garrison, 1997). 

Self-regulated or self-directed strategies enable learners to solve problems in new domains (Ertmer & 
Newby, 1996; Morrow, Sharkey, & Firestone, 1993) or to solve real-world problems (Bolhuis, 1996; 
Temple & Rodero, 1995). For example, in computer-supported science classes, middle-school students 
took more responsibility for their learning, and concurrently, displayed greater competence in complex 
problem-solving strategies (Raghavan, Sartoris, & Glaser, 1997). Another study suggested that 
students who learned in a self-directed environment were more productive. When writers were 
allowed to choose their own topics, they wrote more often and they wrote longer pieces (Morrow et 
al., 1993). 

Academic Achievement 

The ultimate goal of technology immersion is increasing the academic progress of students. Available 
evidence on the effects of laptops on student achievement comes from a few studies that have made 
comparisons between student groups with and without technology. Findings, although limited, have 
generally been positive. 

The strongest evidence on the effects of laptops on achievement is in the area of writing. Lowther, 
Ross, and Morrison (2001, 2003) reported highly significant effects favoring sixth- and seventh-grade 
students with laptops over control students for dimensions of writing, such as ideas and content, 
organization, and style. In a less methodologically rigorous study, Rockman ET AL. (1999) found that 
laptop students outscored non-laptop students on four measures of writing, including content; 
organization; language, voice, and style; and mechanics, conventions, and presentation. 

Some studies also have reported positive effects of one-to-one laptop access on students higher order 
problem solving (Lowther et al., 2003). Evaluation of a laptop project in Beaufort County, West 
Virginia, which focused on outcomes measured by a nationally standardized achievement test, found 
that laptop students participating in the program for two years had higher language, reading, and 
mathematics scores than non-laptop students (Stevenson, 1998). However, since there was no 
statistical control for prior achievement, findings are in doubt. Certainly, additional research studies 
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with experimental designs are needed to draw definitive conclusions about the effects of one-to-one 
initiatives on student achievement. 
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Appendix B 

Survey Items and Scale Reliabilities 


Table B.l. Items and Reliabilities for School-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

Leadership and System Support 

0.91 

0.92 

0.92 

0.94 

The principal consults with staff before making decisions about instructional 
technology that affect us. 





In this school, there are clear expectations that technology will be used to enhance 
student learning. 





The principal in my school actively encourages teachers to pursue professional 
development geared towards curricular integration of technology. 





Our school has a well-developed technology plan that guides all technology 
integration efforts. 





The principal is an effective leader for instructional technology in this school. 





Overall, considering the uses of technology in my school today, 1 am confident that 
this use is leading to increased student achievement. 





The principal encourages teachers to be innovative and try new methods. 





The principal is willing to support through funding or manpower teachers’ efforts at 
technology integration. 





Administrators in this school help teachers to use technology to access, analyze, 
and interpret student performance data. 





Teachers receive adequate administrative support to integrate technology into 
classroom practice. 





Teachers and administrators rely on research-proven teaching and learning 
principles in making decisions about technology use. 





When our school has professional development focused on technology, the 
principal often participates. 





Classroom Technology Integration 

0.67 

0.68 

0.62 

0.76 

Students have adequate access to technology resources in my classroom (e.g., 
digital cameras, scanners, projectors). 





1 incorporate the TEKS for student technology applications into my content-area 
lessons. 





1 have received sufficient training to incorporate technology into my instruction. 





1 use technology to assess student performance and plan instruction. 





Technical Support 

0.71 

0.71 

0.68 

0.66 

Most of our school computers are kept in good working condition. 





Internet connections in my class are often too slow or not working. 





My requests for technical assistance are addressed in a timely manner. 





Materials (e.g., software, printer supplies) for classroom use of computers are 
readily available in my school. 





Problems such as computers freezing or an inability to access the Internet make it 
difficult for me to use technology. 





Innovative Culture 

0.78 

0.79 

0.81 

0.82 

Teachers in this school share an understanding about how technology will be used 
to enhance learning. 





Teachers in this school are continually learning and seeking new ideas. 





Teachers are not afraid to learn about new technologies and use them with their 
class(es). 





Teachers in this school are generally supportive of technology integration efforts. 





Parent and Community Support 

0.78 

0.79 

0.84 

0.85 

Parents support our school's emphasis on technology. 





The surrounding community actively supports our instructional efforts with 
technology. 
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Table B.2. Items and Reliabilities for Teacher-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

Technology Proficiency: 1 am confident that 1 can... 

0.97 

0.97 

0.97 

0.97 

Send email to coworkers, parents, or peers. 





Collaborate through subscribing to a discussion list. 





Create an address book to send email to several people at once. 





Send a document as an attachment to an email message. 





Use a variety of search strategies, including key word and Boolean logic to find Web 
pages related to my subject matter interests. 





Search for and find a Web site with information about the Alamo. 





Create my own World Wide Web home page. 





Keep track of Web sites 1 have visited so that 1 can return to them later. (An 
example is using bookmarks.) 





Find primary sources of information on the Internet that 1 can use in my teaching. 





Use a spreadsheet (e.g., excel) to enter and calculate numbers. 





Use a spreadsheet to create a pie chart. 





Create a newsletter using desktop publishing techniques, including graphics & text 
in 3 columns. 





Perform basic software application functions such as opening an application 
program and creating, modifying, printing, and saving documents. 





Plan, create, and edit documents using word processing software (e.g., Word). 





Use the computer to create a slideshow presentation (e.g., Powerpoint). 





Plan, create, and edit databases using database software (e.g., Access). 





Use a database to search for and sort information and create reports. 





Use graphic organizers and/or systems thinking software (Inspiration, Stella, etc.) to 
teach concepts. 





Use drawing or painting software (e.g., Paint, Illustrator) to create pictures. 





Create a lesson or unit that incorporates subject matter software as an integral part. 





Use technology to collaborate with other colleagues who are distant from my 
classroom. 





Describe 5 software programs that 1 would select and use in my teaching. 





Write a plan with a budget to buy technology for my classroom. 





Teach my students about copyright issues as they relate to the Internet including 
citing sources. 





Take photos with a digital camera, save in a digitized format, and use in an 
electronic document. 





Scan images from a print source such as a book, save them in a digitized format, 
and use them in an electronic document. 





Create products incorporating text, audio, video, and graphics using multimedia 
authoring programs (e.g., Authorware, Plyperstudio). 





Professional Productivity: As a teacher, 1... 

0.93 

0.94 

0.93 

0.94 

Keep administrative records (e.g., attendance). 





Manage student assessment data (e.g., electronic gradebooks). 





Use technology to analyze and interpret student data to guide instruction. 





Create electronic lesson plans. 





Communicate with students. 





Communicate with parents. 





Communicate with colleagues/other professionals. 





Create instructional materials (e.g., tests, handouts). 





Gather information from the internet to create a lesson (e.g., text, video, clipart). 





Access model lesson plans integrating technology. 





Deliver information using presentation software (e.g., Powerpoint). 





Deliver information using multimedia presentations (text, audio, video, graphics). 





Post homework, class requirements, or project information on a website. 





Administer a formative assessment using Texas Mathematics Diagnostic System. 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

Administer other online assessments. 





Use the internet at home for instructional purposes. 





Use a computer to do schoolwork at home. 





Students’ Technology Use: Students in my class use technology to... 

0.95 

0.98 

0.99 

0.98 

Express themselves in writing (e.g., word processing). 





Learn and practice skills (e.g., instructional software or educational games). 





Enter, calculate, and graph information (e.g., Excel spreadsheet). 





Create a database of information for a class project (e.g., Filemaker Pro, Access). 





Create and make presentations (e.g., Powerpoint). 





Communicate by email with peers, experts, or others on topics they are studying. 





Use online discussions to gather information for an assignment (e.g., through 
discussion boards or videoconferencing). 





Conduct internet research on an assigned topic. 





Conduct multimedia research (reference CDs, online encyclopedias). 





Enhance or express conceptual understanding through simulation/modeling 
software. 





Visually represent or investigate concepts (e.g., through concept mapping, 
graphing, reading charts). 





Produce print products (e.g., desktop publishing). 





Produce multimedia reports/projects (e.g., with video, graphics, and sound editing). 





Analyze information using tools such as graphing calculators or digital microscopes. 





Design web sites or web pages. 





Complete a test or quiz (e.g., online assessments, Texas Math Diagnostic System). 





Other (specify) 





Collaboration: As a teacher, 1... 

0.90 

0.92 

0.88 

0.93 

Act as a coach or mentor to other teachers or staff at my school. (May include 
teaching in-service workshop in your school.) 





Receive coaching or mentoring from an external (non-school) source such as a 
professional curriculum developer. 





Receive coaching or mentoring from an internal source, such as another teacher or 
technology coordinator. 





Have informal discussions with colleagues regarding strategies for integrating 
technology. 





Receive feedback from other teachers based on their observations of my teaching. 





Provide feedback to other teachers based on my observations of their teaching. 





Consult with other teachers about certain students' technology skills or use. 





Exchange feedback with other teachers based on student work that used 
technology. 





Work with a subject-area peer to develop a lesson plan or class activity using 
technology. 





Work with a colleague in a different subject area to develop a lesson plan. 





Participate in a study group with other teachers on a technology-related topic. 





Technology Integration 

0.94 

0.95 

0.93 

0.95 

1 alter my instructional use of the classroom computer(s) based upon the newest 
software applications and research on teaching, learning, and standards-based 
curriculum. 





My students discover innovative ways to use classroom computers to make a 
difference in their lives. 





1 allocate time for students to practice their computer skills on the classroom 
computer(s). 





1 integrate the most current research on teaching and learning when using the 
classroom computer(s). 





In my classroom, students use technology-based computer and Internet resources 
beyond the school (NASA, other government agencies, private sector) to solve 
authentic problems. 





My students' authentic problem solving is supported by continuous access to a vast 
array of computer-based tools and technology. 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

1 plan computer-related activities in my classroom that will improve my students’ 
basic skills (e.g., reading, writing, math computation). 





It is easy for me to design student-centered, integrated curriculum units that use the 
classroom computer(s) in a seamless fashion. 





1 seek out activities that promote increased problem-solving and critical thinking 
using the classroom computer(s). 





Using cutting edge technology and computers, 1 have stretched the instructional 
computing in my classroom. 





Learner-Centered Instruction 

0.75 

0.80 

0.79 

0.81 

Students' authentic use of information and inquiry skills guides the type of 
instructional materials used in my classroom. 





My students are involved in establishing individual goals within the classroom 
curriculum. 





In addition to traditional assessments, 1 consistently provide alternative assessment 
opportunities that encourage students to “showcase” their content understanding in 
nontraditional ways. 





My instructional approach emphasizes experiential learning, student involvement, 
and students solving “real-world” issues. 





Resistance to Integration 

0.70 

0.72 

0.81 

0.77 

1 do not find computers to be a necessary part of classroom instruction. 





Using the classroom computer(s) is not a priority for me this school year. 





1 do not find the use of computers to be practical for my students. 
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Table 6.3. Items and Reliabilities for Student-Level Scales 


Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

Technology Proficiency: How far along are you in learning to... 

0.94 

0.94 

0.94 

0.94 

open, create, modify, print, and save documents 





use a digital camera and/or scanner to get pictures into the computer 





send a document as an attachment to an email 





keep track of Web sites 1 have visited so that 1 can return to them later (using 
bookmarks, etc.) 





enter information on the computer using proper keyboarding skills 





gather information from CD-ROMS 





use online reference databases (online encyclopedias, newspapers, Library of 
Congress, etc.) to gather information 





use a search engine to find information about a topic (Alamo, etc.) on the Web 





narrow Web searches using key words and Boolean logic (such as “or,” “and,” or 
“not”) 





use online discussions with experts or mentors to gather information 





evaluate information found on the Web for accuracy 





use a word processor (AppleWorks, Word, etc.) to write and print a story or report 





use a spreadsheet (AppleWorks, Excel, etc.) to enter and calculate numbers 





use a spreadsheet to create graphs 





use a database (AppleWorks, Access, etc.) to enter information 





use a database to search for and sort information and create reports 





use software (Keynote, PowerPoint, etc.) to create a presentation 





use drawing or painting software (Paint, Illustrator, etc.) to create pictures 





use a video camera to make a video 





use software (HyperStudio, Authorware, etc.) to create a multimedia product 





use email to send and receive messages 





use software (Frontpage, Publisher, etc.) to create web pages 





Technology Use in School: In your English language arts, mathematics, social 
studies, and science classes, how often do your teachers have you... 

0.90 

0.92 

0.93 

0.91 

use a word processor (AppleWorks, Word, etc.) to write a story or report. 





use software to learn and practice skills (Riverdeep, Compass Learning, PLATO 
Learning, etc.). 





use a spreadsheet (Excel, etc.) to enter and calculate numbers or create graphs 
for an assignment. 





create a database of information (Filemaker Pro, Access, etc.) for a class project. 





create a presentation (PowerPoint, etc.) and present information to classmates or 
others. 





communicate by email with friends, experts, and others about topics you are 
studying. 





use online discussions to gather information for an assignment (discussion 
boards, videoconferencing, etc.). 





conduct Internet research on an assigned topic. 





use tools, such as graphing calculators or digital microscopes, to analyze 
information. 





produce print products (with desktop publishing software). 





create multimedia reports or projects (with video, graphics, and sound editing). 





use technology to complete a test or quiz. 





Other 





Technical Problems 

0.83 

0.85 

0.86 

0.84 

The computer is broken or slow. 





The program 1 need is not on the computer. 





The Internet connection is too slow or not working. 





A website 1 need is blocked by a filter. 





Sharing a computer makes it hard to finish assignments. 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

My teacher can't fix things when something goes wrong. 





Other (describe) 





Small-Group Work: When students work together in small groups in my classes, we... 

0.80 

0.83 

0.81 

0.83 

review and give advice on each other’s work. 





tutor or coach each other on difficult work. 





make a presentation for the rest of the class. 





brainstorm solutions to problems. 





discuss previous class assignments. 





produce a report or project. 





School Satisfaction 

0.77 

0.82 

0.78 

0.80 

1 am satisfied with the work that 1 do in my classes. 





1 understand why 1 am doing the things we do in my classes. 





The things we do in my classes will help me as an adult. 





The work we do in my classes will be useful to me in the job 1 hope to have as an 
adult. 





1 work hard in my classes because the work is meaningful. 





What 1 learn in my classes is more important than the grade 1 receive. 





Self-Directed Learning 

0.88 

0.89 

0.88 

0.89 

If I’m confused in class, 1 ask the teacher or another student for help. 





Sometimes, if 1 think an assignment is too tough, 1 purposely don't try hard. Then 
if 1 don't do well, 1 don't feel bad. 





At the end of a project or assignment, I'll think about how hard 1 worked and 
whether 1 would do anything differently next time. 





It's important to me that 1 understand my schoolwork really well. 





Even when 1 think my schoolwork is boring, 1 keep working until I'm finished. 





Before 1 begin studying, 1 think about or list the things I'm going to do during my 
study time. 





Even when I'm supposed to learn about something boring, 1 keep working until 1 
finish. 





When my teacher writes comments on assignments, 1 don't read them unless 1 
have to. 





When we start a new unit, 1 like to know what we're going to be learning and how 
I'll know if I've learned it well. 





When the teacher calls on me, and 1 make a mistake in class, 1 can honestly say 
that 1 don't feel bad. 





When 1 do well on a big project, it's because I've worked hard. 





1 work harder than 1 need to on my schoolwork, because that’s just the way 1 am. 





I'll recopy my notes or make diagrams of what we're learning to try and remember 
it better. 





1 don't like asking for help with my schoolwork. 





If a topic is too hard, it’s really hard for me to stay motivated. 





If 1 know I'm going to do badly on a task, 1 try to avoid it, even if 1 know I’d learn a 
lot from it. 





There are some subjects I'm just bad at. 





A lot of times, I'll wait until the last minute to do my homework or study for a test. 





1 know 1 can make a schedule to get my work done on time and stick to it. 





When I'm doing homework, 1 rush to finish if 1 have ,a friend coming over or if a 
good TV show is about to start. 





I'll look through mistakes 1 made on earlier assignments so 1 don't make the same 
mistakes on new assignments. 





When I'm done writing a report, 1 read it over carefully and think 0 about whether 
I've done a good job. 





Even if 1 try, 1 can't make myself concentrate on schoolwork when there are more 
interesting things to do. 





When I'm reading a chapter, 1 ask myself questions to make sure 1 understand the 
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Scale/ 

Item 

Fall 

2004 

Cronbach’s Alpha 
Spring Fall 
2005 2005 

Spring 

2006 

material. 





There are some subjects 1 just can't understand, even if 1 try hard. 





When 1 get a bad grade, 1 feel dumb. 





I'll pick a tough project where 1 would learn a lot over an easy project, even if it 
means I'll have to work harder to get a good grade 





This happens to me a lot: I'll study for a test and think 1 understand everything; 
then 1 take the test and don't do very well. 





1 don't really take notes when I'm reading something for school. 





When 1 get a grade 1 don’t like, I'll spend time trying to figure out what 1 could have 
done differently. 





When 1 do badly on a project, 1 feel okay as long as 1 did better than some of the 
other kids in my class. 





When 1 answer a question wrong in class, 1 end up wishing I'd never spoken up. 





When 1 get a bad grade, it's because 1 could have studied more or because 1 
should have done something differently, like taking better notes. 





If I’m having trouble concentrating, 1 find a place to study where 1 won't be 
distracted. 





The things we're learning in my class are usually really interesting. 





If 1 have to choose, I'd rather get good grades in a class than learn a lot. 





When a big project or report is assigned, 1 make a mental or written schedule to 
make sure everything gets done on time. 





I’ll usually ask someone (like my parents, friends or teacher) to give me feedback 
on my ideas when I’m working on a big assignment. 





1 know from past experience exactly what 1 have to do (like schedule a certain 
amount of time, or take notes in a particular way) if 1 want to do well on my 
sohoolwork. 





If an assignment isn't going to count toward my grade, 1 don't need to know how 
well 1 did on it. 





1 only feel bad about a low grade if 1 think 1 didn't work hard enough, or if 1 think 1 
made careless mistakes 





When 1 read, 1 put the important ideas into my own words. 





When I'm not feeling motivated, 1 can't, make myself study. 





When 1 don't understand things in class, 1 end up thinking it's because I'm not that 
smart. 





When we have a reading assignment, I'll read through it one time, but 1 don't really 
go back through it to check how well 1 remember it. 





1 know 1 can do well in school if 1 try hard enough. 





1 don't ask for help, even if 1 don't understand the directions for an assignment. 





1 wouldn't do any homework if 1 didn't have to. 
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Appendix C 

Measuring Implementation Fidelity 


In the second year, we employed a two-part approach to the measurement of implementation fidelity. 
First, we used indicators to describe each campus’ progress on a 4-step scale toward immersion 
standards. Rating scales for components and related elements identified four levels of immersion: 
minimal (0 to 1.99), partial (2.00 to 2.99), substantial (3.00 to 3.49), and full (3.50 to 4.00). Second, 
we used quantitative implementation indices that gauged the level of technology immersion using 
standardized scores (z scores). Both the immersion standard scores and implementation indices were 
derived from values for seven components: (a) Leadership, (b) Teacher Support, (c) Parent and 
Community Support, (d) Technical Support, (e) Professional Development, (f) Classroom Immersion, 
and (g) Student Access and Use. The following sections describe the seven components of technology 
immersion and related measurement procedures. Table C.l shows the scoring rubrics for immersion 
indicators, and Table C.2 describes the data sources used to generate scores. 

Supports for Implementation 

Leadership 

Our measure of principal leadership comes from teacher survey items (12) that yield a Leadership 
scale score. Items assess the extent to which the principal involves staff in decisions, sets clear 
expectations for technology use, encourages and participates in professional development, has a well- 
developed technology plan, promotes teacher innovation, and provides necessary resources and 
administrative support. Teachers rated the extent of their agreement on a 5-point scale ranging from 0 
{strongly disagree ) to 4 {strongly agree). To achieve substantial to full immersion, teachers had to 
agree or strongly agree that the principal provided technology leadership. A Leadership Index was 
generated by transforming the scale score to a z score. 

Teacher Support 

Although implementation may be affected by the characteristics of individual teachers, it also may 
reflect the collective disposition of teachers toward the adoption of new and innovative practices. Our 
measure of teacher commitment to technology immersion comes from teacher survey items (4) 
measuring a Teacher Support scale (i.e.. Innovative Culture). Items gauged the extent to which 
teachers in the school share an understanding about technology use for student learning, are 
continually learning and seeking new ideas, arc not afraid to learn about and use new technologies, 
and are generally supportive of technology integration efforts. Teachers rated the extent of their 
agreement on a 5-point scale ranging from 0 ( strongly disagree ) to 4 {strongly agree), with substantial 
to full immersion tied to the strength of teacher agreement. A Teacher Support Index was generated by 
transforming the scale score to a z score. 

Parent and Community Support 

Support from parents and community members is also a key paid of implementation because they must 
understand the goals of technology immersion, assume responsibility along with their children, and 
assist in enacting effective policies. Our measure of Parent and Community Support is a scale score 
composed of teacher survey items (2). These items indicate the extent to which parents support the 
school’s emphasis on technology and the community actively supports instructional efforts with 
technology. Teachers rated the extent of their agreement on a 5-point scale ranging from 0 {strongly 
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disagree ) to 4 ( strongly agree). Substantial to full immersion reflected the strength of teacher 
agreement. A Parent/Community Support Index was generated by transforming the scale score to a z 
score. 

Technical Support 

On a fully immersed campus, sufficient technical support and a healthy infrastructure are expected to 
alleviate technical problems that might interfere with the use of technology in the classroom, school, 
and beyond. Our measure for technical support comes from teacher survey items (5) contributing to a 
Technical Support scale score. Teachers indicated the extent of their agreement on a 5-point scale 
ranging from 0 ( strongly disagree) to 4 ( strongly agree) that computers are kept in good working 
order, requests for assistance are addressed in a timely way, Internet connections work adequately, and 
classroom materials are readily available. A Technical Support Index was generated by transforming 
the scale score to a z score. 

Professional Development 

In constructing measures of professional development, we drew from research conducted on the 
effectiveness of the Eisenhower Professional Development Program (e.g., Caret, Porter, Desimone, 
Birman, & Yoon, 2001). Key features of quality professional development provided a framework for 
examining dimensions of schools’ and vendors’ professional development models. Data for measures 
come from core-subject teachers’ responses to survey items. 

First, we measured the total number of Contact Hours that core-subject teachers spent in technology- 
related professional development during the two-year technology immersion implementation period. 

In addition, professional development models for technology immersion were required to include a 
classroom support component, so we measured Classroom Support as the extent to which core 
teachers indicated that they received modeling, coaching or mentoring from an internal source (such as 
another teacher or technology coordinator), or an external source (such a professional curriculum 
developer). Teachers rated the frequency of support on a 4-point scale linked to standards: 0 {never), 
1.33 ( rarely — a few times a year), 2.67 ( sometimes — once or twice a month), and 4 ( often — once or 
twice a week or almost daily). 

To examine the Content Focus of teachers’ activities, we asked each teacher who participated in 
technology-related professional development to indicate the degree of emphasis the activity placed on 
curriculum, instructional methods, and lesson development in their core-subject area. Teachers’ 
responses were coded on a 5-point scale with 0 = no emphasis, 2 = minor emphasis, and 4 -major 
emphasis. As a measure of professional development Coherence, each core teacher who attended 
technology-related events indicated the extent to which the activity was consistent with the their goals 
for professional development, was based explicitly on what the teacher had learned in earlier 
professional development experiences, was followed up with activities that built on what the teacher 
learned in the professional development activity, was aligned with state or district standards and 
curriculum frameworks and with state and district assessments. To measure this indicator, teachers 
used a 5-point scale ranging from 0 ( not at all) to 4 {to a great extent). A Professional Development 
Index was generated by averaging z scores for each of the four professional development elements. 
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Extent of Implementation 
Classroom Immersion 

The technology immersion packages included a variety of instructional and assessment resources 
designed to extend, supplement, or enhance core-subject teaching and learning. Wireless laptops, for 
example, were loaded with productivity software (i.e., either Appleworks or Microsoft Office ) for 
students to use as a learning tool. Teachers and students also received a variety of digital resources and 
formative assessments to support content-area instruction and learning activities. Indicators for 
Classroom Immersion, accordingly, assessed the extent to which core-subject teachers at immersion 
campuses utilized resources and embraced practices consistent with the technology immersion model. 
Classroom Immersion is measured by five elements: Technology Integration, Learner-Centered 
Instruction, Student Classroom Activities, Communication, and Professional Productivity. Measures 
of Technology Integration (10 items) and Learner-Centered Instruction (4 items) are scale scores 
adapted from the Levels of Technology Implementation (LoTi) Questionnaire. Core teachers indicated 
the extent to which statements related to Technology Integration (e.g., I alter my instructional practices 
to support higher order thinking through technology) and Learner-Centered Instruction (e.g., I have 
students use information and inquiry skills) are hue on a 5-point scale, including 0 ( not true of me 
now), 1 to 3 ( somewhat true of me now), and 4 (very true of me now). 

Because teachers influence students’ classroom opportunities to use technology for learning academic 
content, we also used items from teacher surveys as a way to assess the extent to which teachers had 
students use various technology applications in core-subject classrooms (Student Classroom 
Activities). For example, survey items gauged how often students' used a word processor to write a 
story or used software to learn and practice skills. Teachers’ responses were converted to a 5-point 
scale tied to immersion standards. Responses indicated how often students’ in a typical class used 
technology in particular ways: 0 (never), 1.33 (rarely — a few times a year), 2.67 (sometimes — once or 
twice a month), 4.00 (often — once or twice a week — or almost daily). 

Teachers at immersion schools also are expected to use technology as a communication tool. 
Communication that advances student learning involves sending email to students, parents, or 
colleagues, or posting information and assignments on a class or school website. Technology also 
provides a way to improve teachers’ Professional Productivity, including the use of technology for 
purposes such as keeping records, analyzing data, developing lessons, or delivering information. Scale 
scores for Communication (4 items) and Professional Productivity (11 items) are comprised of teacher 
responses on a 5-point scale indicating the frequency of activities: 0 (never) to 4.00 (almost daily). The 
Classroom Immersion Index was generated by averaging z scores for each of the five elements 
described above. 

Student Access and Use 

This indicator gauged the extent of student access to laptop computers as well as the frequency of 
students’ laptop use for learning in core-content classrooms and at home. Three elements — Laptop 
Access Days, Core-Content Learning, and Home Learning — contribute to the component score. First, 
in an immersion school, students are expected to have access to wireless laptops for the entire school 
year. Our measure of Laptop Access was calculated as the number of days out of the 1 80-day school 
year that students actually had laptops available for use. Information for the indicator comes from an 
analysis of student survey items in which students indicated whether the school provided a laptop for 
student use, and if provided, how many days the laptop had been taken away (e.g., for misuse, 
misbehavior, failure to complete assignments, bad grades, or repairs). Student access scores, which 
could range from 0 days (no laptop) to 180 days (laptop available the full school year), were converted 
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to the 0-4.00 continuous scale to measure progress toward the immersion standard. A Laptop Access 
Index was generated by transforming the continuous score to a z score. 

The potential for laptops to affect achievement depends largely on students’ opportunities to use 
technology for learning core academic content. Consequently, we used items from student surveys (4) 
to assess the frequency with which students used technology resources in their English/language arts, 
mathematics, science, and social studies classrooms (Core-Content Learning). Students’ responses 
were converted to a 4-point frequency scale tied to standards: 0 ( never or rarely — a few times a year), 
1.33 ( sometimes — once or twice a month), 2.67 ( often — once or twice a week), and 4 ( almost daily). A 
Core-Content Learning Index was generated by transforming the scale score to a z score. 

Additionally, on a fully immersed campus, students should have access to their wireless laptops for 
learning both within and outside of school. Information for the measure of Home Learning comes 
from student survey items in which students indicated whether the school provided a laptop for student 
use, how often the student could take a laptop home, and if a laptop could be taken home, how often it 
was used for homework in core subjects or for learning games. A student’s use of the laptop for home 
learning was rated on a 6-point scale: 0 (no access to laptop outside of school), 1 ( restricted or full 
access to laptop outside of school), plus up to 5 additional points if a student used their laptop for 
homework in ELA, math, science, or social studies, or for learning games. Students’ scores were 
converted to the 0-4.00 scale as a measure of progress toward immersion standards, and a z score was 
generated. We generated the Student Access and Use Index by averaging z scores for each of the three 
elements described above. 
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Table C.l. Scoring Rubrics for Measuring the Implementation Fidelity of Technology Immersion (Continued) 
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Table C.2. Data Sources for Technology Immersion Implementation Indicators 
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Table C.2. Data Sources for Technology Immersion Implementation Indicators (Continued) 
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Table C.2. Data Sources for Technology Immersion Implementation Indicators (Continued) 
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Table C.2. Data Sources for Technology Immersion Implementation Indicators (Continued) 
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Appendix D 

Effects of Technology Immersion on Schools 


On one part of surveys, teachers responded to items pertaining to their perceptions of school-level 
supports for technology. Teachers were asked to rate the strength of their agreement with statements 
on a 5-point scale ranging from 1 ( strongly disagree ) to 5 ( strongly agree). Four distinct 
organizational factors emerged from a factor analysis: Leadership (12 items). Innovative Culture 
(4 items). Parent and Community Support (2 items), and Technical Support (5 items). Cronbach’s 
alpha reliability coefficients for the scale scores ranged from acceptable (0.66) to excellent (0.94). 

In addition to using school-level supports for technology to measure the implementation levels of the 
technology immersion model, we also have used school-level scores to compare treatment and control 
teachers prior to technology immersion and after the first and second implementation years. We 
analyzed the effects of immersion on teachers’ perceptions of school-level indicators of technology 
support using separate t tests to estimate differences between immersion and control teachers. We also 
calculated effect sizes (Cohen’s d) as a way to show the relative strength of differences. Effect size 
(ES) are expressed as standard deviations. For example, an effect size of 1.00 indicates teachers’ 
average score at an immersion school is one full standard deviation above the scores for control group 
teachers. Effect sizes can be interpreted generally as large (greater than 0.50), moderate (0.50-0.30), 
small (0.30-0.10), or trivial (less than 0.10). 

Effects on School-Level Technology Supports 

In Chapter 3, we reported that the strength of school-level supports for technology immersion varied 
across the 21 treatment campuses. Collectively, however, teachers at immersion schools perceived 
significantly stronger technology supports than teachers at control campuses. Data from analyses of 
school variables, summarized in Table D.l, show that at the end of both the first implementation year 
(spring 2005) and second implementation year (spring 2006), teachers at immersion schools reported 
significantly higher scale scores than control teachers on item measuring their levels of agreement 
with the provision of four school-level supports for technology: 

• Leadership (Effect sizes = 0.44 and 0.33, respectively, in spring 2005 and 2006), 

• Teacher Support (Effect sizes = 0.44 and 0.40), 

• Parent and Community Support (Effect sizes = 0.51 and 0.24), and 

• Technical Support (Effect sizes = 0.33 and 0.20). 

Teachers’ responses across time help to explain the effects of immersion on their perceptions of 
technology-related supports. 

Leadership 

The scale score for Leadership indicates the extent to which teachers believe their principal establishes 
a clear vision and expectations for technology, encourages classroom integration, provides needed 
supports, and involves staff in decision making. Results show that in the fall of 2004, teachers at 
immersion schools reported significantly higher Leadership scores for their principals than control 
teachers (M = 3.71 vs. 3.60, ES = 0.17). This may have reflected teachers’ initial optimism about their 
principals’ proactive efforts to secure Technology Immersion Pilot (TIP) grants. In the spring of 2005, 
after one year of project implementation, immersion teachers reported even stronger Leadership (ES = 


123 




0.44), and their higher estimations of principal leadership, compared to control teachers, was sustained 
at the end of the second year (ES = 0.35). Teachers at immersion schools tended to agree that their 
principals provided technology-related support. 


Table D.l. Comparison Group Differences for School-Level Technology Support Variables 



Immersion 

Control 





N = 

=21 

N= 

=21 



Effect 

School Variables 

Mean 

SD 

Mean 

SD 

/-value 

P 

Size 

Leadership 






Fall 2004 

3.71 

0.61 

3.60 

0.67 

2.88 

0.004* 

0.17 

Spring 2005 

3.89 

0.61 

3.60 

0.68 

6.91 

0.000* 

0.44 

Spring 2006 

3.89 

0.65 

3.64 

0.75 

5.78 

0.000* 

0.35 

Teacher Support (for innovation) 






Fall 2004 

3.72 

0.66 

3.70 

0.67 

0.42 

0.673 

0.03 

Spring 2005 

4.00 

0.60 

3.73 

0.65 

6.91 

0.000* 

0.44 

Spring 2006 

4.06 

0.62 

3.80 

0.68 

6.62 

0.000* 

0.40 

Parent and Community Support 






Fall 2004 

3.44 

0.78 

3.39 

0.74 

1.16 

0.248 

0.07 

Spring 2005 

3.75 

0.71 

3.38 

0.74 

8.16 

0.000* 

0.51 

Spring 2006 

3.64 

0.82 

3.44 

0.85 

3.93 

0.000* 

0.24 

Technical Support 






Fall 2004 

3.29 

0.74 

3.30 

0.76 

-0.33 

0.742 

-0.02 

Spring 2005 

3.60 

0.69 

3.36 

0.76 

5.19 

0.000* 

0.33 

Spring 2006 

3.63 

0.67 

3.49 

0.70 

3.35 

0.001* 

0.20 


Notes. Scale scores range from 1.00 to 5.00. Fall 2004: N= 521 to 530 immersion teachers, /V=604 to 609 control teachers; 
spring 2005: 7V=465 to 478 immersion teachers and N=52A to 544 control teachers; spring 2006: N= 505 to 517 immersion 
teachers and N=576 to 586 control teachers. *Statistically significant difference. Effect size is Cohen's d. The effect size is 


interpreted as follows: a value greater than 0.5 is large, 0.5-0. 3 is moderate, 0.3-0. 1 is small, and anything smaller than 0.1 is 
trivial. 


Innovative Culture 

The scale score for Innovative Culture reflects the extent to which teachers at a school share an 
understanding about technology use, continually learn, are unafraid of new technologies, and are 
generally supportive of technology integration efforts. In fall 2004, there were no significant 
differences between groups in teachers’ opinions. However, at the end of each implementation year, 
teachers at immersion schools were significantly more likely than control teachers to view their school 
culture as innovative (ES = 0.44 and 0.40, respectively, in 2005 and 2006). Immersion teachers 
generally agreed with statements reflecting an Innovative Culture for technology. 

Parent and Community Support 

Scores for Parent and Community Support show the extent to which teachers believe that parents and 
the surrounding community support the school’s efforts with technology. There were no significant 
differences between immersion and control teachers’ views on the degree of support from parents and 
community members in fall of 2004, but at the end of the first implementation year (spring 2005), 
teachers at immersion schools reported a significantly stronger level of support than control teachers 
(ES = 0.50). After the second implementation year (spring 2006), immersion teachers’ positive 
perceptions of Parent and Community Support had waned, but their estimations of support still 
significantly exceeded control teachers’ views (ES = 0.24). Even so, immersion teachers expressed a 
degree of uncertainty about Parent and Community Support for immersion at the end of the second 
year. 
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Technical Support 

The Technical Support scale indicates the extent to which teachers believe technical problems with 
computers, Internet access, repairs, and material availability pose barriers to technology integration. In 
fall 2004, there were no significant differences between immersion and con tied teachers’ perceived 
support. However, with the infusion of technology resources and additional support staff through 
Technology Immersion Pilot (TIP) grants, immersion teachers reported significantly higher levels of 
Technical Support than control teachers at the end of the first and second implementation years (ES’s 
= 0.33 and 0.22, respectively, in 2005 and 2006). 

In sum, teachers at immersion schools compared to control perceived stronger school-level 
organizational supports for technology from principals, parents and the community, and technical 
staff. And, as a whole, they also expressed greater affinity for innovative technology practices. 
Notably, however, teachers at control schools reported increasingly higher levels of agreement for 
each of the organizational support indicators across survey years. An additional, longitudinal analysis 
for teachers who were in the immersion project from fall 2004 through spring 2006 showed significant 
increases in immersion teachers’ reported agreement with the extent of support for each of the four 
indicators (similar to the cross-sectional findings). Longitudinal results for control teachers who were 
at the schools during the same time period revealed statistically significant increases for two school- 
level indicators (Technical Support and Innovative Culture). 
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Appendix E 

Technical Appendix — Hierarchical Linear Modeling (HLM) 


Effects of Technology Immersion on Teachers and Teaching (Chapter 4) 

Researchers estimated the effects of immersion on teacher mediating variables using three-level 
hierarchical linear growth models. In our models, we posit that school poverty is related to teachers’ 
initial status and yearly growth rate. Statistical details are provided in Tables E.l, E.2, and E.3. The 
models’ simplicity aids in the interpretation of effects. More complex models, controlling for teacher 
demographic characteristics (gender, ethnicity, experience), described subsequently in Tables E.4, E.5 
and E.6, estimated nearly identical immersion growth coefficients. 


Table E.l . Descriptive Statistics for Teacher Variables: 
HLM Models with School Poverty 


1 Variable Name 

N 

Mean 

SD ; 

Repeated Measures Descriptive Statistics (Level 1) 

Survey Time 

2,406 

1.00 

0.82 

Technology Proficiency 

2,311 

4.65 

1.43 

Professional Productivity 

2,294 

3.15 

0.72 

Technology Integration 

2,192 

3.49 

1.53 

Learner-Centered Instruction 

2,255 

3.92 

1.34 

Resistance to Integration 

2,275 

2.31 

1.30 

Student Classroom Activities 

2,283 

2.10 

0.79 

Collaboration 

2,291 

2.49 

0.78 

School-Level Descriptive Statistics (Level 3) 1 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School percent economically disadvantaged 

42 

68.58 

17.56 


Table E.2. Immersion (Fixed) Effect Analyses of Teacher Mediating Variables: 
HLM Models with School Poverty 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

4.476 

0.086 

51.88*** 


Immersion dummy 

-0.151 

0.143 

-1.06 


School poverty 

- 0.001 

0.004 

-0.15 


Growth rate 

0.176 

0.033 

5.26*** 


Immersion dummy 

0.178 

0.044 

4.02*** 


School poverty 

-0.004 

0.001 

- 2 . 66 * 

Professional Productivity 


Initial status (fall 2004) 

2.984 

0.056 

53.30*** 


Immersion dummy 

-0.057 

0.080 

-0.72 


School poverty 

0.002 

0.002 

0.68 


Growth rate 

0.089 

0.020 

4 49 *** 


Immersion dummy 

0.167 

0.028 

5.90*** 


School poverty 

- 0.001 

0.001 

- 1.11 

Technology Integration 


Initial status (fall 2004) 

2.836 

0.068 

41.80*** 


Immersion dummy 3 

0.234 

0.116 

2 . 02 * _ 


School poverty 

0.009 

0.003 

2.83** 


Growth rate 

0.265 

0.044 

6.08*** 


Immersion dummy 3 

0.528 

0.074 

7 jo*** 


School poverty 

-0.004 

0.002 

- 1 .74f 


Continued 
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Table E.2. Immersion (Fixed) Effect Analysis of Teacher Mediating Variables (Continued) 


1 Learner-Centered Instruction ! 


Initial status (fall 2004) 

3.561 

0.051 

69.45*** 


Immersion dummy 

0.034 

0.090 

0.37 


School poverty 

0.005 

0.002 

2.30* 


Growth rate 

0.200 

0.037 

5.39*** 


Immersion dummy 

0.278 

0.066 

4 20 *** 


School poverty 

-0.002 

0.002 

-1.18 

Resistance to Integration 


Initial status (fall 2004) 

2.359 

0.054 

43.59*** 


Immersion dummy b 

-0.240 

0.075 

-3.20** 


School poverty 

0.001 

0.002 

0.76 


Growth rate 

0.083 

0.038 

2.20* 


Immersion dummy b 

-0.092 

0.055 

-1.66 


School poverty 

0.000 

0.002 

-0.08 

Student Classroom Activities 


Initial status (fall 2004) 

1.864 

0.045 

41.03*** 


Immersion dummy 

0.052 

0.065 

0.80 


School poverty 

0.004 

0.002 

2.07* 


Growth rate 

0.043 

0.021 

2.01* 


Immersion dummy 

0.285 

0.036 

8.03*** 


School poverty 

0.000 

0.001 

0.13 

Collaboration 


Initial status (fall 2004) 

2.255 

0.053 

42.24*** 


Immersion dummy 0 

0.138 

0.080 

1 ,73 + 


School poverty 

0.003 

0.002 

1.28 


Growth rate 

0.055 

0.022 

2.47* 


Immersion dummy 0 

0.154 

0.042 

3.72** 


School poverty 

0.001 

0.001 

1.14 


f p < .10; *p < .05; **p < .01; ***/> < .001. 

a Immersion teachers had significantly higher initial technology integration scores. A latent variable regression, controlling for the 
effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion 
coefficients was not significant (the difference divided by the standard error of the difference = 0.43). 

b Immersion teachers had significantly lower initial resistance to integration scores. A latent variable regression, controlling for the 
effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion 
coefficients was not significant (the difference divided by the standard error of the difference = 0.42). 

c Immerssion teachers had significantly higher initial levels of collaboration. A latent variable regression, controlling for the effect of 
this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion coefficients was 
not significant (the difference divided by the standard error of the difference = 1.54). 
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Table E.3. Variance Decomposition from Conditional HLM Growth Models of Teacher 
Mediating Variables (with School Poverty) 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.2926 




Level-2 individual initial status 

1.8021 

760 

6219.50 

0.000 

Level-2 individual growth rate 

0.0723 

760 

1126.72 

0.000 

Level-2 school initial status 

0.0720 

39 

75.76 

0.001 

Level-2 school growth rate 

0.0064 

39 

70.54 

0.002 

Professional Productivity 

Level- 1 temporal variation 

0.1451 




Level-2 individual initial status 

0.3213 

755 

2658.38 

0.000 

Level-2 individual growth rate 

0.0113 

755 

864.43 

0.004 

Level-2 school initial status 

0.0332 

39 

102.17 

0.000 

Level-2 school growth rate 

0.0028 

39 

74.06 

0.001 

Technology Integration 

Level- 1 temporal variation 

0.6095 




Level-2 individual initial status 

1.3284 

741 

2465.51 

0.000 

Level-2 individual growth rate 

0.0349 

741 

816.25 

0.028 

Level-2 school initial status 

0.0222 

39 

60.30 

0.016 

Level-2 school growth rate 

0.0302 

39 

104.73 

0.000 

Learner-Centered Instruction 

Level- 1 temporal variation 

0.6904 




Level-2 individual initial status 

1.0164 

751 

1989.28 

0.000 

Level-2 individual growth rate 

0.0406 

751 

835.96 

0.016 

Level-2 school initial status 

0.0054 

39 

53.82 

0.057 

Level-2 school growth rate 

0.0171 

39 

73.77 

0.001 

Resistance to Integration 

Level- 1 temporal variation 

0.7286 




Level-2 individual initial status 

0.7968 

749 

1711.66 

0.000 

Level-2 individual growth rate 

0.0801 

749 

919.71 

0.000 

Level-2 school initial status 

0.0075 

39 

44.58 

0.248 

Level-2 school growth rate 

0.0064 

39 

54.49 

0.051 

Student Classroom Activities 

Level- 1 temporal variation 

0.1920 




Level-2 individual initial status 

0.3025 

753 

2101.71 

0.000 

Level-2 individual growth rate 

0.0134 

753 

845.06 

0.011 

Level-2 school initial status 

0.0136 

39 

69.90 

0.002 

Level-2 school growth rate 

0.0051 

39 

81.68 

0.000 

Collaboration 

Level- 1 temporal variation 

0.2287 




Level-2 individual initial status 

0.2626 

753 

1743.19 

0.000 

Level-2 individual growth rate 

0.0178 

753 

867.25 

0.003 

Level-2 school initial status 

0.0334 

39 

100.76 

0.000 

Level-2 school growth rate 

0.0081 

39 

85.22 

0.000 
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Researchers also used HLM growth models to estimate immersion effects on teacher mediating 
variables, controlling for teacher characteristics. Statistical details for these models are provided in 
Tables E.4, E.5 and E.6. 

Table E.4. Descriptive Statistics for Teacher Variables: 

HLM models with Teacher Characteristics 


1 Variable Name 

N 

Mean 

SD 1 

Repeated Measures Descriptive Statistics (Level 1) 

Time 

2,406 

1.00 

0.82 

Technology Proficiency 

2,311 

4.65 

1.43 

Professional Productivity 

2,294 

3.15 

0.72 

Technology Integration 

2,192 

3.49 

1.53 

Learner-Centered Instruction 

2,255 

3.92 

1.34 

Resistance to Integration 

2,275 

2.31 

1.30 

Student Classroom Activities 

2,283 

2.10 

0.79 

Collaboration 

2,291 

2.49 

0.78 

Teacher-Level Descriptive Statistics (Level 2) 

Male 

802 

0.30 

0.46 

Hispanic 

802 

0.39 

0.49 

African American 

802 

0.04 

0.20 

Experience 

802 

12.76 

9.29 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 


Table E.5. Immersion (Fixed) Effect Analyses of Teacher-Level Variables: 
HLM Models with Teacher Characteristics 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 


Initial status (fall 2004) 

5.281 

0.089 

59 19 *** 


Immersion dummy 

- 0.112 

0.112 

- 1.00 


Male 

-0.125 

0.130 

-0.96 


Hispanic 

-0.249 

0.097 

-2.57) 


African American 

0.200 

0.132 

1.51 


Experience 

-0.055 

0.006 

-9 81*** 


Growth rate 

0.098 

0.035 

2.76** 


Immersion dummy 

0.181 

0.048 

3.75** 


Male 

-0.054 

0.041 

-1.31 


Hispanic 

0.010 

0.052 

0.19 


African American 

0.033 

0.065 

0.50 


Experience 

0.006 

0.002 

2 77 ** 

Professional Productivity 


Initial status (fall 2004) 

3.277 

0.080 

40 77*** 


Immersion dummy 

-0.047 

0.074 

-0.63 


Male 

-0.167 

0.064 

-2.60* 


Hispanic 

0.044 

0.053 

0.84 


African American 

0.113 

0.087 

1.29 


Experience 

- 0.020 

0.004 

-4.68*** 


Growth rate 

0.044 

0.026 

1.73* 


Immersion dummy 

0.167 

0.028 

(3 04 *** 


Male 

- 0.012 

0.025 

-0.49 


Hispanic 

0.004 

0.024 

0.19 


African American 

0.046 

0.066 

0.70 


Experience 

0.003 

0.001 

2.30* 


(Continued) 
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Table E.5. Immersion (Fixed) Effect Analysis of Teacher-Level Variables (Continued) 


1 Technology Integration j 


Initial status (fall 2004) 

3.069 

0.112 

27.35*** 


Immersion dummy 

0.235 

0.125 

1.89 


Male 

-0.140 

0.095 

-1.47 


Hispanic 

0.335 

0.111 

3.02** 


African American 

0.556 

0.178 

3.12** 


Experience 

-0.023 

0.006 

-3.88*** 


Growth rate 

0.277 

0.056 

4 94 *** 


Immersion dummy 

0.535 

0.077 

6.93*** 


Male 

-0.089 

0.043 

-2.07* 


Hispanic 

-0.118 

0.065 

-1.82 


African American 

0.093 

0.101 

0.92 


Experience 

0.003 

0.002 

1.15 

Learner-Centered Instruction 


Initial status (fall 2004) 

3.498 

0.062 

56.83*** 


Immersion dummy 

0.054 

0.093 

0.58 


Male 

-0.144 

0.089 

-1.62 


Hispanic 

0.256 

0.103 

2.49* 


African American 

0.642 

0.160 

4 01*** 


Experience 

-0.025 

0.005 

_4 94 *** 


Growth rate 

0.237 

0.038 

6.28*** 


Immersion dummy 

0.284 

0.067 

4 25 *** 


Male 

- 0.112 

0.057 

-1.95* 


Hispanic 

-0.043 

0.058 

-0.74 


African American 

0.082 

0.090 

0.92 


Experience 

0.003 

0.003 

1.09 

Resistance to Integration j 


Initial status (fall 2004) 

2.080 

0.087 

23.84*** 


Immersion dummy 3 

-0.263 

0.070 

-3.76*** 


Male 

0.416 

0.097 

4 29 *** 


Hispanic 

-0.228 

0.065 

-3.50** 


African American 

-0.599 

0.094 

-6.36*** 


Experience 

0.023 

0.005 

4 93 *** 


Growth rate 

0.125 

0.052 

2.38* 


Immersion dummy 3 

-0.094 

0.056 

-1.69 


Male 

0.034 

0.062 

0.55 


Hispanic 

0.024 

0.056 

0.43 


African American 

- 0.011 

0.073 

-0.15 


Experience 

-0.005 

0.002 

- 2 . 22 * 

Student Classroom Activities 


Initial status (fall 2004) 

1.868 

0.071 

26.34*** 


Immersion dummy 

0.062 

0.061 

1.01 


Male 

-0.034 

0.080 

-0.42 


Hispanic 

0.250 

0.059 

4 20 *** 


African American 

0.449 

0.083 

5 4 j *** 


Experience 

-0.007 

0.004 

-1.71 


Growth rate 

0.044 

0.024 

1.82* 


Immersion dummy 

0.285 

0.036 

7 93 *** 


Male 

0.022 

0.043 

0.50 


Hispanic J 

-0.016 

0.027 

-0.60 


African American 

-0.014 

0.073 

-0.19 


Experience 

0.000 

0.002 

-0.03 


(Continued) 
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Table E.5. Immersion (Fixed) Effect Analysis of Teacher-Level Variables (Continued) 


1 Collaboration j 

Initial status (fall 2004) 

2.298 

0.068 

34.02*** 

Immersion dummy b 

0.147 

0.075 

1.96* 

Male 

0.034 

0.059 

0.58 

Hispanic 

0.127 

0.056 

2.28* 

African American 

0.502 

0.091 

5.53*** 

Experience 

-0.008 

0.003 

-2.60* 

Growth rate 

0.061 

0.033 

1.86* 

Immersion dummy b 

0.152 

0.041 

3.70** 

Male 

-0.049 

0.021 

-2.38* 

Hispanic 

0.034 

0.030 

1.15 

African American 

-0.122 

0.066 

-1.85 

Experience 

0.000 

0.001 

0.36 


*p < .10; *p < .05; **p < .01; ***p < .001. 

a Immersed teachers had significantly lower initial resistance to integration scores. A latent variable regression was used to control for 
this initial difference. This analysis indicated that there was not a significant immersion effect on resistance to integration growth 
rate ( t for immersion dummy = 0.23, p = 0.817). 

b Immersed teachers had significantly higher initial levels of collaboration scores. A latent variable regression was used to control for 
this initial difference. This analysis indicated that there was still a significant immersion effect on collaboration growth rate ( t for 
immersion dummy = 5.88, p = 0.001). The difference between the original and adjusted coefficients was not significant ( t = 1.70). 
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Table E.6. Variance Decomposition from Conditional HLM Growth Models of Teacher 
Mediating Variables (with Teacher Characteristics) 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.2927 




Level-2 individual initial status 

1.5736 

756 

5429.65 

0.000 

Level-2 individual growth rate 

0.0694 

756 

1153.92 

0.000 

Level-2 school initial status 

0.0217 

40 

50.63 

0.121 

Level-2 school growth rate 

0.0091 

40 

75.69 

0.001 

Professional Productivity 

Level- 1 temporal variation 

0.1454 




Level-2 individual initial status 

0.2832 

751 

2243.76 

0.000 

Level-2 individual growth rate 

0.0105 

751 

906.61 

0.000 

Level-2 school initial status 

0.0271 

40 

95.02 

0.000 

Level-2 school growth rate 

0.0026 

40 

72.75 

0.001 

Technology Integration 

Level- 1 temporal variation 

0.6090 




Level-2 individual initial status 

1.2483 

737 

2431.18 

0.000 

Level-2 individual growth rate 

0.0296 

737 

829.10 

0.010 

Level-2 school initial status 

0.0419 

40 

61.51 

0.016 

Level-2 school growth rate 

0.0337 

40 

107.74 

0.000 

Learner-Centered Instruction 

Level- 1 temporal variation 

0.6902 




Level-2 individual initial status 

0.9411 

747 

2012.51 

0.000 

Level-2 individual growth rate 

0.0362 

747 

867.13 

0.002 

Level-2 school initial status 

0.0103 

40 

54.57 

0.062 

Level-2 school growth rate 

0.0179 

40 

74.97 

0.001 

Resistance to Integration 

Level- 1 temporal variation 

0.7280 




Level-2 individual initial status 

0.7004 

786 

1687.90 

0.000 

Level-2 individual growth rate 

0.0754 

745 

952.49 

0.000 

Level-2 school initial status 

Effect Not Random 

Level-2 school growth rate 

0.0108 

40 

66.87 

0.005 

Student Classroom Activities 

Level- 1 temporal variation 

0.1921 




Level-2 individual initial status 

0.2862 

749 

2091.54 

0.000 

Level-2 individual growth rate 

0.0132 

749 

890.97 

0.000 

Level-2 school initial status 

0.0104 

40 

62.24 

0.014 

Level-2 school growth rate 

0.0051 

40 

82.15 

0.000 

Collaboration 

Level- 1 temporal variation 

0.2289 




Level-2 individual initial status 

0.2482 

749 

1669.44 

0.000 

Level-2 individual growth rate 

0.0164 

749 

903.43 

0.000 

Level-2 school initial status 

0.0282 

40 

93.19 

0.000 

Level-2 school growth rate 

0.0083 

40 

88.06 

0.000 
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Effects of Technology Immersion on Students and Learning (Chapter 5) 


For the results reported in Chapter 5, researchers analyzed the effects of immersion on student 
mediating variables for Cohorts 1 and 2 using two- and three-level FILM models. 

Effects on Mediating Variables 

Cohort 1 . Researchers’ used three-level FILM growth models, with controls for school poverty 
(percentage of economically disadvantaged students) and student poverty (qualification for free- or 
reduced-price lunch). The models’ simplicity aids in the interpretation of effects. Statistical details are 
provided in Tables E.7, E.8, and E.9 for analyses of mediating variables for Cohort 1. We analyzed 
more complex growth models, including controls for other student demographic variables (gender, 
ethnicity, home Internet access). Since these models yielded immersion growth coefficients nearly 
identical to those reported in Tables E.7, E.8 and E.9, results are not provided in this report. 


Table E.7. Descriptive Statistics for Student Variables, Cohort 1 


1 Variable Name 

N 

Mean 

SD 1 

Repeated Measure Descriptive Statistics (Level 1] 

> 1 

Time 

9,300 

1.00 

0.82 

Technology Proficiency score 

9,054 

3.13 

0.90 

Technology Use in School score 

8,423 

2.29 

0.83 

Technical Problems score 

8.717 

2.42 

0.93 

Small-Group Work score 

8,565 

2.79 

0.87 

School Satisfaction score 

8,713 

3.69 

0.75 

Self-Directed Learning score 

8,263 

4.48 

0.75 

Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

3,100 

0.73 

0.45 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.58 

17.56 


Table E.8. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 1 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 






Initial status (fall 2004) 

3.031 

0.058 

51.89*** 


Immersion dummy 

-0.037 

0.084 

-0.44 


School poverty 

0.000 

0.003 

-0.14 


Disadvantaged 

-0.361 

0.040 

_9 12 *** 


Growth rate 

0.210 

0.026 

8.08*** 


Immersion dummy 

0.198 

0.044 

4.53*** 


School poverty 

-0.001 

0.00 1_ 

-0.52 


Disadvantaged 

0.060 

0.017 

3.49** 

Self-Directed Learning 






Initial status (fall 2004) 

4.551 

0.042 

107.67*** 


Immersion dummy b 

0.105 

0.051 

2.08* 


School poverty 

0.003 

0.00 1_ 

2.26* 


Disadvantaged 

-0.039 

0.037 

-1.07 


Growth rate 

-0.068 

0.019 

-3.59** 


Immersion dummy b 

-0.025 

0.024 

-1.04 


School poverty 

-0.001 

0.001 

-0.82 


Disadvantaged 

-0.029 

0.018 

-1.60 


(i Continued ) 
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Table E.8. Immersion (Fixed) Effect Analysis of Student Variables, Cohort 1 (Continued) 


School Satisfaction 






Initial status (fall 2004) 

3.807 

0.041 

92.69*** 


Immersion dummy 

0.047 

0.040 

1.18 


School poverty 

0.000 

0.001 

- 0.21 


Disadvantaged 

-0.091 

0.032 

-2.89** 


Growth rate 

-0.094 

0.019 

_4 99 *** 


Immersion dummy 

0.016 

0.023 

0.72 


School poverty 

0.001 

0.001 

2.23* 


Disadvantaged 

0.033 

0.015 

2.18* 

Classroom Activities 





(with technology) 

Initial status (fall 2004) 

2.039 

0.062 

33.02*** 


Immersion dummy 

0.133 

0.08 1_ 

1.64 


School poverty 

- 0.002 

0.002 

-0.72 


Disadvantaged 

0.005 

0.030 

0.17 


Growth rate 

0.031 

0.026 

1.17 


Immersion dummy 

0.323 

0.059 

5.46*** 


School poverty 

0.000 

0.002 

0.09 


Disadvantaged 

0.058 

0 . 02 1 _ 

2.83** 

Small-Group Work 






Initial status (fall 2004) 

2.826 

0.055 

51.64*** 


Immersion dummy 

0.001 

0.062 

0.01 


School poverty 

- 0.002 

0.002 

-0.90 


Disadvantaged 

-0.049 

0.056 

- 0.86 


Growth rate 

-0.105 

0.027 

-3.89** 


Immersion dummy 

0.173 

0.043 

4.05*** 


School poverty 

0.000 

0.001 

- 0.20 


Disadvantaged 

0.063 

0.034 

1 . 86 f 

Technical Problems 






Initial status (fall 2004) 

2.362 

0.059 

39 . 94 *** 


Immersion dummy 2 

-0.140 

0.075 

- 1 . 88 f 


School poverty 

-0.003 

0.003 

-1.17 


Disadvantaged 

-0.078 

0.059 

-1.33 


Growth rate 

0.100 

0.030 

3.30** 


Immersion dummy 2 

0.084 

0.054 

1.55 


School poverty 

0.001 

0.002 

0.36 


Disadvantaged 

0.060 

0.029 

2.08* 


f p < .10; *p < .05; **p < .01; ***p < .001. 


a Immersion students had borderline significantly lower initial technical problems scores. A latent variable regression, controlling for the 
effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion coefficients 
was not significant (the difference divided by the standard error of the difference = 1.76). 

b Immersion students had significantly higher initial technical self-directed learning scores. A latent variable regression, controlling for the 
effect of this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion coefficients 
was not significant (the difference divided by the standard error of the difference = 1.44). 
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Table E.9. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 1 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 temporal variation 

0.2934 



Level-2 individual initial status 

0.4521 

304 1_ 

8525.47 

0.000 

Level-2 individual growth rate 

0.0367 

304 1_ 

3790.18 

0.000 

Level-2 school initial status 

0.0636 

39 

293.64 

0.000 

Level-2 school growth rate 

0.0172 

39 

351.94 

0.000 

Self-Directed Learning 

Level- 1 temporal variation 

0.2267 



Level-2 individual initial status 

0.3362 

2883 

7618.66 

0.000 

Level-2 individual growth rate 

0.0250 

2883 

3477.81 

0.000 

Level-2 school initial status 

0.0152 

39 

122.12 

0.000 

Level-2 school growth rate 

0.0030 

39 

102.41 

0.000 

School Satisfaction 

Level- 1 temporal variation 

0.3363 



Level-2 individual initial status 

0.2142 

3005 

5190.24 

0.000 

Level-2 individual growth rate 

0.0182 

3005 

3378.01 

0.000 

Level-2 school initial status 

0.0078 

39 

84.87 

0.000 

Level-2 school growth rate 

0.0022 

39 

70.22 

0.002 

Classroom Activities 

Level- 1 temporal variation 

0.4228 



Level-2 individual initial status 

0.1416 

3050 

5838.58 

0.000 

Level-2 individual growth rate 

Effect Not Random 

Level-2 school initial status 

0.0565 

39 

261.98 

0.000 

Level-2 school growth rate 

0.0330 

39 

345.21 

0.000 

Small-Group Work 

Level- 1 temporal variation 

0.5094 



Level-2 individual initial status 

0.2503 

2968 

4597.65 

0.000 

Level-2 individual growth rate 

0.0358 

2968 

3394.22 

0.000 

Level-2 school initial status 

0.0252 

39 

124.48 

0.000 

Level-2 school growth rate 

0.0128 

39 

146.56 

0.000 

Technical Problems 

Level- 1 temporal variation 

0.5268 



Level-2 individual initial status 

0.2547 

3012 

4679.02 

0.000 

Level-2 individual growth rate 

0.0620 

3012 

3700.39 

0.000 

Level-2 school initial status 

0.0396 

39 

210.58 

0.000 

Level-2 school growth rate 

0.0229 

39 

252.61 

0.000 
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Cohort 2. Researchers’ used two-level HLM models, with controls for fall scale scores, student 
demographic characteristics (gender, ethnicity, economic disadvantage), and school poverty 
(percentage of economically disadvantaged students) and student poverty (qualification for free- or 
reduced-price lunch), to estimate the effects of immersion on student mediating variables for Cohort 2. 
Statistical details for are provided in Tables E.10, E.l 1, and E.12. 


Table E.10. Descriptive Statistics for Student Mediating Variables, Cohort 2 


1 Variable Name 

N 

Mean 

SD 1 

Student-Level Descriptive Statistics (Level 1) 

Female 

4,021 

0.50 

0.50 

Hispanic 

4,020 

0.68 

0.47 

African American 

4,020 

0.07 

0.26 

Eco. disadvantaged (1 = yes, 0 = no) 

4,019 

0.73 

0.44 

Technology Proficiency (fall 2005) 

3,820 

2.84 

0.89 

Technology Proficiency (spring 2006) 

3,917 

3.25 

0.84 

Self-Directed Learning (fall 2005) 

3,364 

4.67 

0.74 

Self-Directed Learning (spring 2006) 

3,586 

4.54 

0.70 

School Satisfaction (fall 2005) 

3,612 

3.76 

0.75 

School Satisfaction (spring 2006) 

3,808 

3.72 

0.74 

Classroom Activities (fall 2005) 

3,564 

2.09 

0.85 

Classroom Activities (spring 2006) 

3,669 

2.45 

0.80 

Small-Group Work (fall 2005) 

3,538 

2.75 

0.91 

Small-Group Work (spring 2006) 

3.743 

2.81 

0.86 

Technical Problems (fall 2005) 

3,653 

2.04 

0.88 

Technical Problems (spring 2006) 

3,769 

2.37 

0.87 

School-Level Descriptive Statistics (Level 2) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percentage) 

42 

68.58 

17.56 
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Table E.11. Immersion (Fixed) Effect Analyses of Student Mediating Variables, Cohort 2 


School-Level Scale 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Technology Proficiency 






Base 

3.229 

0.054 

60.01*** 


Immersion dummy 

0.251 

0.071 

3.52** 


School poverty 

0.001 

0.002 

0.71 


Female 

0.074 

0.018 

4 19*** 


Hispanic 

-0.064 

0.028 

-2.31* 


African American 

- 0.111 

0.048 

-2.29* 


Disadvantaged 

-0.125 

0.035 

-3.63** 


Fall 2005 score 

0.477 

0.023 

20.72*** 

Self-Directed Learning 






Base 

4.593 

0.029 

155.85*** 


Immersion dummy 

-0.023 

0.032 

-0.73 


School poverty 

0.001 

0.001 

1.69 + 


Female 

0.030 

0.026 

1.15 


Hispanic 

-0.029 

0.029 

-0.98 


African American 

0.037 

0.041 

0.90 


Disadvantaged 

-0.027 

0.026 

-1.03 


Fall 2005 score 

0.556 

0.020 

27.70*** 

School Satisfaction 






Base 

3.688 

0.033 

110.37*** 


Immersion dummy 

-0.008 

0.033 

-0.23 


School poverty 

- 0.001 

0.001 

- 1.20 


Female 

0.113 

0.026 

4 44 *** 


Hispanic 

0.068 

0.030 

2.28* 


African American 

0.073 

0.073 

1.00 


Disadvantaged 

-0.092 

0.027 

-3.42** 


Fall 2005 score 

0.345 

0.023 

15.23*** 

Classroom Activities 





(with technology) 

Base 

2.118 

0.061 

34 77 *** 


Immersion dummy 

0.663 

0.092 

7 i7*** 


School poverty 

0.003 

0.003 

1.03 


Female 

-0.003 

0.023 

-0.13 


Hispanic 

0.048 

0.036 

1.33 


African American 

0.142 

0.045 

3.15** 


Disadvantaged 

0.014 

0.034 

0.42 


Fall 2005 score 

0.234 

0.014 

16.55*** 

Small-Group Work 






Base 

2.610 

0.057 

45.61*** 


Immersion dummy 

0.214 

0.067 

3.17** 


School poverty 

0.001 

0.002 

0.26 


Female 

0.096 

0.033 

2.91** 


Hispanic 

0.092 

0.045 

2.06* 


African American 

0.197 

0.052 

3.75*** 


Disadvantaged 

-0.009 

0.039 

-0.24 


Fall 2005 score 

0.227 

0.020 

11.65*** 

Technical Problems 






Base 

2.361 

0.059 

39 92 *** 


Immersion dummy 

0.046 

0.076 

0.61 


School poverty 

0.005 

0.002 

2 . 22 * 


Female 

0.013 

0.028 

0.49 


Hispanic 

-0.055 

0.037 

-1.49 


African American 

0.074 

0.064 

1.17 


Disadvantaged 

-0.043 

0.031 

-1.38 


Fall 2005 score 

0.202 

0.027 

7 42 *** 


f p < .10; *p < .05; **p < .01; ***p < .001. 
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Table E.12. Variance Decomposition from Conditional HLM Models of Student Mediating Variables, 
Cohort 2 


Scale/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Technology Proficiency 

Level- 1 student effect 

0.4322 




School mean 

0.0526 

39 

364.79 

0.000 

School pretest-outcome slope 

0.0108 

41 

102.77 

0.000 

Self-Directed Learning 

Level- 1 student effect 

0.3231 




School mean 

0.0066 

39 

87.46 

0.000 

School pretest-outcome slope 

0.0058 

41 

62.04 

0.018 

School Satisfaction 

Level- 1 student effect 

0.4621 




School mean 

0.0046 

39 

65.92 

0.005 

School pretest-outcome slope 

0.0088 

41 

74.04 

0.001 

Classroom Activities 

Level- 1 student effect 

0.4734 




School mean 

0.0895 

39 

591.95 

0.000 

School pretest-outcome slope 

Effect Not Random 

Small-Group Work 

Level- 1 student effect 

0.6505 




School mean 

0.0456 

39 

194.02 

0.000 

School pretest-outcome slope 

0.0041 

41 

56.88 

0.050 

Technical Problems 

Level- 1 student effect 

0.6827 




School mean 

0.0494 

39 

280.58 

0.000 

School pretest-outcome slope 

Effect Not Random 


Effects on School Attendance 

Comparable to analyses for student-level valuables, we used three-level HLM growth models and two- 
level HLM analyses to estimate the effects of immersion on student attendance. Statistical details are 
provided in Tables E.13, E.14, and E.15. 


Table E.13. Descriptive Statistics for Student Attendance 


1 Variable Name 

N 

Mean 

SD 1 

Cohort 1 Repeated Measures Descriptive Statistics (Level 1) 

Year 

13,194 

1.00 

0.82 

Attendance 

13,094 

96.68 

4.00 

Cohort 1 Student-Level Descriptive Statistics (Level 2) 

Eco. disadvantaged (1 = yes, 0 = no) 

4,398 

0.74 

0.44 ] 

Cohort 1 School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty 

42 

68.58 

17.56 

Cohort 2 Student-Level Descriptive Statistics (Level 1) 

Attendance 2006 

4,398 

96.11 

4.61 

Attendance 2004 

4,298 

97.24 

3.47 

Eco. disadvantaged (1 = yes, 0 = no) 

4,019 

0.73 

0.44 

Hispanic (1 = yes, 0 = no) 




African American (1 = yes, 0 = no) 




Lernale (1 = yes, 0 = no) 




Cohort 2 School-Level Descriptive Statistics (Level 2) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty 

42 

68.58 

17.56 
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Table E.14. Immersion (Fixed) Effect Analyses of Student Attendance 


Group 

School-Level 

Analysis 

Gamma 

Coefficient 

Standard 

Error 

t 

Cohort 1 






Initial attendance (2004) 

97.076 

0.157 

616.69*** 


Immersion dummy 

-0.053 

0.224 

-0.24 


School poverty 

0.022 

0.005 

4 41 *** 


Eco. disadvantaged 

-0.702 

0.132 

-5.31*** 


Growth rate 

-0.355 

0.067 

-5 27*** 


Immersion dummy 

-0.217 

0.100 

-2.18* 


School poverty 

-0.005 

0.003 

-1.84 f 


Eco. disadvantaged 

-0.154 

0.082 

- 1 . 86 f 

Cohort 2 






Base 

96.922 

0.186 

520.22*** 


Immersion dummy 2 

-0.268 

0.146 

-1.83 f 


School poverty 

0 . 00 1 _ 

0.006 

0.25 


Prior attendance 

0.627 

0.030 

20.73*** 


Eco. disadvantaged 

-0.560 

0.146 

-3.83*** 


Female 

0.242 

0.134 

1 .8 1 f 


Hispanic 

0.115 

0.208 

0.55 


African American 

-0.329 

0.386 

-0.85 


< .10; *p < .05; **p < .01; ***p < .001. 


Table E.15. Variance Decomposition from Conditional HLM Models of Student attendance, 
Cohorts 1 and 2 


Cohort/ 
Random Effect 

Variance 

Component 

df 

X 2 

Pt 

Cohort 1 

Level- 1 temporal variation 

4.5052 




Level-2 individual initial status 

6.7248 

4355 

12027.70 

0.000 

Level-2 individual growth rate 

1.9582 

4355 

8105.03 

0.000 

Level-2 school initial status 

0.3679 

39 

180.00 

0.000 

Level-2 school growth rate 

0.0487 

39 

94.78 

0.000 

Cohort 2 

Level- 1 student effect 

11.4126 




School mean 

0.1040 

39 

70.388 

0.002 

School pre-measure-outcome slope 

0.0236 

41 

137.933 

0.000 
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Effects of Technology Immersion on Student Achievement (Chapter 6) 


Researchers used two-level HLM models and three-level HLM growth models to estimate the effects 
of immersion on student academic achievement. Statistical details arc provided for Cohort 1 students 
(seventh graders) in Tables E. 16 through E.19, and for Cohort 2 students (sixth graders) in Tables E.20 
through E.22. 

Cohort 1 (Seventh Graders) 

Table E.16. Descriptive Statistics for TAKS Reading and Mathematics, Cohort 1 


1 Variable Name 

N 

Mean 

SD 1 

Repeated Measures Descriptive Statistics: Reading (Level 1) 

Time 

10,257 

1.00 

0.82 

TAKS Reading T score 

10,257 

48.28 

9.82 

Repeated Measures Descriptive Statistics: Mathematics (Level 1) 

Time 

10,350 

1.00 

0.82 

TAKS Mathematics T score 

10,350 

48.40 

9.53 

Student-Level Descriptive Statistics 

: Reading (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,419 

0.70 

0.46 

Student-Level Descriptive Statistics 

: Mathematics (Level 2) 


Eco. disadvantaged (1 = yes, 0 = no) 

3,450 

0.71 

0.46 

School-Level Descriptive Statistics (Level 3) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percent) 

42 

68.58 

17.56 


Table E.17. Descriptive Statistics for TAKS Writing, Cohort 1 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

: Writing (Level 1) 


Female 

3,126 

0.52 

0.50 

African American 

3,126 

0.06 

0.24 

Hispanic 

3,126 

0.68 

0.47 

Eco. disadvantaged (1 = yes, 0 = no) 

3,126 

0.70 

0.46 

TAKS Writing T score (2003) 

3,126 

47.46 

8.10 

TAKS Writing T score (2006) 

3,126 

49.23 

9.23 

School-Level Descriptive Statistics (Level 2) 1 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percent) 

42 

68.58 

17.56 
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Table E.18. Immersion (Fixed) Effect Analyses of TAKS Achievement, Cohort 1 


School-Level Gamma Standard 

TAKS Achievement Test Analysis Coefficient Error r-value 

Reading 

Initial status (spring 2004) 53.834 0.635 84.75*** 

Immersion dummy' 1 -1.350 0.649 -2.08* 

School poverty -0.064 0.014 -4.71*** 

Eco. disadvantaged -6.384 0.666 -9.58*** 

Growth rate -0.547 0.190 -2.87** 

Immersion dummy 2 0.257 0.246 1.05 

School poverty 0.007 0.008 0.84 

Eco. disadvantaged 0.461 0.191 2.41* 


Mathematics 



Initial status (spring 2004) 

52.813 

0.659 

80.17*** 


Immersion dummy 2 

-1.074 

0.829 

-1.30 


School poverty 

-0.047 

0.018 

-2.61* 


Eco. disadvantaged 

-4.558 

0.506 

-9.01*** 


Growth rate 

-0.315 

0.265 

-1.19 


Immersion dummy 2 

0.220 

0.445 

0.49 


School poverty 

0.008 

0.012 

0.61 


Eco. disadvantaged 

-0.285 

0.149 

-1.91' 


Writing 



Base 

51.116 

0.650 

78.70*** 


Immersion dummy 

-0.907 

0.616 

-1.47 


School poverty 

-0.011 

0.017 

-0.63 


Female 

2.237 

0.356 

6.29*** 


African American 

-2.241 

0.553 

-4.05*** 


Hispanic 

-1.162 

0.345 

-3.37** 


Eco. disadvantaged 

-1.712 

0.507 

-3.38** 


Spring 2003 T score 

0.617 

0.020 

30.28*** 


'p < .10; < .05; **p < .01; ***/? < .001. 

a Immersion students had significantly lower initial TAKS reading scores. A latent variable regression, controlling for the effect of 
this initial difference on the growth rate, indicated that the difference between the original and adjusted immersion coefficients 
was not significant (the difference divided by the standard error of the difference = 0.26). 


Table E.19. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 1 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 temporal variation 

22.5847 




Level-2 individual initial status 

55.1410 

3376 

13268.14 

0.000 

Level-2 individual growth rate 

1.3225 

3376 

3773.09 

0.000 

Level-2 school initial status 

2.8666 

39 

160.85 

0.000 

Level-2 school growth rate 

0.4114 

39 

141.44 

0.000 

Mathematics 

Level- 1 temporal variation 

20.3428 




Level-2 individual initial status 

57.0502 

3407 

14868.56 

0.000 

Level-2 individual growth rate 

0.7548 

3407 

3660.25 

0.002 

Level-2 school initial status 

5.6630 

39 

233.15 

0.000 

Level-2 school growth rate 

1.8953 

39 

485.15 

0.000 

Writing 

Level- 1 student effect 

47.4347 




School mean 

3.4650 

39 

191.57 

0.000 

School pre-measure-outcome slope 

0.0054 

41 

59.64 

0.030 
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Cohort 2 (Sixth Graders) 


Table E.20. Descriptive Statistics for TAKS Achievement, Cohort 2 


Variable Name 

N 

Mean 

SD 

Student-Level Descriptive Statistics 

: Reading (Level 1) 


Female 

3,789 

0.52 

0.50 

African American 

3,789 

0.07 

0.26 

Hispanic 

3,789 

0.67 

0.47 

Eco. disadvantaged (1 = yes, 0 = no) 

3,789 

0.71 

0.45 

TAKS Reading T score (2005 ) 

3,789 

48.59 

9.54 

TAKS Reading T score (2006) 

3,789 

48.01 

9.71 

Student-Level Descriptive Statistics 

: Mathematics (Level 1) 


Female 

3,838 

0.52 

0.50 

African American 

3,838 

0.07 

0.26 

Hispanic 

3,838 

0.67 

0.47 

Eco. disadvantaged (1 = yes, 0 = no) 

3,838 

0.71 

0.45 

TAKS Mathematics T score (2005) 

3,838 

48.50 

9.37 

TAKS Mathematics T score (2006) 

3,838 

47.39 

9.18 

School-Level Descriptive Statistics i 

[Level 2) 

Immersion status (1 = yes, 0 = no) 

42 

0.50 

0.51 

School poverty (percent) 

42 

68.58 

17.56 


Table E.21. Immersion (Fixed) Effect Analyses of Cohort 2 Achievement 


School-Level 

TAKS Achievement Test Analysis 

Gamma 

Coefficient 

Standard 

Error 

t-value 

Reading 

Base 

48.828 

0.450 

108.41*** 

Immersion dummy 

0.236 

0.428 

0.55 

School poverty 

-0.025 

0.013 

-1.9F 

Female 

1.518 

0.213 

7 jl*** 

African American 

-0.957 

0.361 

-2.65** 

Hispanic 

-0.869 

0.388 

-2.24* 

Eco. disadvantaged 

-1.218 

0.355 

-3.43** 

Spring 2005 T score 

0.648 

0.027 

24.03*** 

Immersion dummy 

0.011 

0.033 

0.34 

Mathematics 

Base 

48.067 

0.495 

97.03*** 

Immersion dummy 

0.950 

0.751 

1.27 

School poverty 

-0.020 

0.018 

-1.11 

Female 

0.816 

0.229 

3.56** 

African American 

-1.961 

0.516 

-3.80*** 

Hispanic 

-0.752 

0.298 

-2.52* 

Eco. disadvantaged 

-1.027 

0.257 

-4 00*** 

Spring 2005 T score 

0.680 

0.021 

31.69*** 

Immersion dummy 

0.050 

0.025 

1.97* 

< .10; *p < .05; **p < .01; ***p < .001. 
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Table E.22. Variance Decomposition from Conditional HLM Models of Student Achievement, 
Cohort 2 


Test/ 

Random Effect 

Variance 

Component 

df 

X 2 

P 

Reading 

Level- 1 student effect 

45.3648 




School mean 

1.3513 

39 

113.32 

0.000 

School pre-measure-outcome slope 

0.0041 

40 

62.77 

0.012 

Mathematics 

Level- 1 student effect 

34.0634 




School mean 

5.6026 

39 

560.78 

0.000 

School pre-measure-outcome slope 

0.0020 

40 

52.44 

0.090 
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